[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:07:57 dist-test-slave-6j0v krb5kdc[17798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:07:57 dist-test-slave-6j0v krb5kdc[17798](info): set up 2 sockets
Sep 27 11:07:57 dist-test-slave-6j0v krb5kdc[17798](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 11:07:58.296444 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:07:58.409488 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:45223
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:07:58.488184 17811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:07:58.488404 17811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:07:58.488425 17811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:07:58.489617 17811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:07:58.489660 17811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:07:58.489674 17811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:07:58.489686 17811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:07:58.489739 17811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:07:58.491247 17811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:07:58.491490 17811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:07:58.491722 17811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:07:58.496652 17817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:07:58.496659 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:07:58.496873 17816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:07:58.496690 17811 server_base.cc:1034] running on GCE node
I20260927 11:07:58.497205 17811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:07:58.497539 17811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:07:58.498737 17811 hybrid_clock.cc:648] HybridClock initialized: now 1790507278498716 us; error 34 us; skew 500 ppm
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:07:58.502128 17811 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:07:58.502635 17811 webserver.cc:460] Webserver started at http://127.17.95.254:35331/ using document root <none> and password file <none>
I20260927 11:07:58.502964 17811 fs_manager.cc:359] Metadata directory not provided
I20260927 11:07:58.503026 17811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:07:58.503190 17811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:07:58.504676 17811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "2d20671acaa6413db8b804a03731d968"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "103584245f7592f1f8f778a1cbb88471"
server_key_iv: "79e6480a92f229f6271698be9e37f298"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.505115 17811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "2d20671acaa6413db8b804a03731d968"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "103584245f7592f1f8f778a1cbb88471"
server_key_iv: "79e6480a92f229f6271698be9e37f298"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.506898 17811 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260927 11:07:58.507741 17826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.507949 17811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:07:58.508044 17811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "2d20671acaa6413db8b804a03731d968"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "103584245f7592f1f8f778a1cbb88471"
server_key_iv: "79e6480a92f229f6271698be9e37f298"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.508114 17811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:07:58.525398 17811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:07:58.527379 17811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:07:58.527501 17811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:07:58.531148 17811 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:45223
I20260927 11:07:58.531183 17878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:45223 every 8 connection(s)
I20260927 11:07:58.531560 17811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:07:58.532387 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:07:58.533900 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 17811
I20260927 11:07:58.533984 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:07:58.534584 17791 external_mini_cluster.cc:1428] Setting key 3a1fae0e755fb8dbd2dd528be192ae5b
I20260927 11:07:58.534740 17879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968: Bootstrap starting.
I20260927 11:07:58.535429 17879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968: Neither blocks nor log segments found. Creating new log.
I20260927 11:07:58.535763 17879 log.cc:824] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968: Log is configured to *not* fsync() on all Append() calls
I20260927 11:07:58.536468 17879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968: No bootstrap required, opened a new log
I20260927 11:07:58.538031 17879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d20671acaa6413db8b804a03731d968" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 45223 } }
I20260927 11:07:58.538184 17879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:07:58.538247 17879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d20671acaa6413db8b804a03731d968, State: Initialized, Role: FOLLOWER
I20260927 11:07:58.538385 17879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d20671acaa6413db8b804a03731d968" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 45223 } }
I20260927 11:07:58.538487 17879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:07:58.538532 17879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:07:58.538578 17879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:07:58.539527 17879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d20671acaa6413db8b804a03731d968" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 45223 } }
I20260927 11:07:58.539644 17879 leader_election.cc:302] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d20671acaa6413db8b804a03731d968; no voters:
I20260927 11:07:58.539808 17879 leader_election.cc:288] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:07:58.539831 17884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:07:58.539963 17884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [term 1 LEADER]: Becoming Leader. State: Replica: 2d20671acaa6413db8b804a03731d968, State: Running, Role: LEADER
I20260927 11:07:58.540060 17879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:07:58.540130 17884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d20671acaa6413db8b804a03731d968" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 45223 } }
I20260927 11:07:58.540484 17884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d20671acaa6413db8b804a03731d968. Latest consensus state: current_term: 1 leader_uuid: "2d20671acaa6413db8b804a03731d968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d20671acaa6413db8b804a03731d968" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 45223 } } }
I20260927 11:07:58.540490 17886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d20671acaa6413db8b804a03731d968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d20671acaa6413db8b804a03731d968" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 45223 } } }
I20260927 11:07:58.540658 17884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [sys.catalog]: This master's current role is: LEADER
I20260927 11:07:58.540669 17886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968 [sys.catalog]: This master's current role is: LEADER
I20260927 11:07:58.540895 17893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:07:58.541637 17893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:07:58.543790 17893 catalog_manager.cc:1347] Generated new cluster ID: 3aaa940e92bb46d58890bea2cec3d71f
I20260927 11:07:58.543853 17893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:07:58.550552 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:58.535460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.17.95.254:45223)
0927 11:07:58.535663 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 11:07:58.535672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:07:58.535906 (+ 234us) server_negotiation.cc:407] Connection header received
0927 11:07:58.538016 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:07:58.538022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:07:58.538067 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:07:58.538203 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:07:58.540093 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.541481 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.542754 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.543370 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.543496 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:58.546929 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:07:58.546932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:07:58.546933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:07:58.546944 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:07:58.548710 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:58.549156 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:58.549159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:58.549159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:58.549260 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:58.549520 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:58.549521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:58.549521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:58.549752 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:07:58.549924 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 11:07:58.550213 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 11:07:58.550346 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":72,"threads_started":1}
I20260927 11:07:58.562342 17893 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:07:58.563041 17893 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:07:58.571823 17893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d20671acaa6413db8b804a03731d968: Generated new TSK 0
I20260927 11:07:58.571998 17893 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:07:58.602618 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:45223
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:07:58.683116 17905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:07:58.683312 17905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:07:58.683337 17905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:07:58.684485 17905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:07:58.684531 17905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:07:58.684588 17905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:07:58.686012 17905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:45223
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:07:58.686278 17905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:07:58.686528 17905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:07:58.691351 17911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:07:58.691351 17910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:07:58.691340 17913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:07:58.691747 17905 server_base.cc:1034] running on GCE node
I20260927 11:07:58.691953 17905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:07:58.692239 17905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:07:58.693423 17905 hybrid_clock.cc:648] HybridClock initialized: now 1790507278693409 us; error 24 us; skew 500 ppm
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:07:58.695988 17905 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:07:58.696441 17905 webserver.cc:460] Webserver started at http://127.17.95.193:42197/ using document root <none> and password file <none>
I20260927 11:07:58.696676 17905 fs_manager.cc:359] Metadata directory not provided
I20260927 11:07:58.696743 17905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:07:58.696893 17905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:07:58.697885 17905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "9b9d67cfb65d41fc8665387f96d32548"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "a180e894749bcf7e3f52fdd133b4fafa"
server_key_iv: "c892cba7fcc215cc7600c24f4979beae"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.698269 17905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "9b9d67cfb65d41fc8665387f96d32548"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "a180e894749bcf7e3f52fdd133b4fafa"
server_key_iv: "c892cba7fcc215cc7600c24f4979beae"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.699772 17905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:07:58.700562 17920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.700750 17905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:07:58.700856 17905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "9b9d67cfb65d41fc8665387f96d32548"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "a180e894749bcf7e3f52fdd133b4fafa"
server_key_iv: "c892cba7fcc215cc7600c24f4979beae"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.700946 17905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:07:58.715605 17905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:07:58.717754 17905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:07:58.717921 17905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:07:58.718149 17905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:07:58.718489 17905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:07:58.718524 17905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.718569 17905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:07:58.718597 17905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.724665 17905 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:44285
I20260927 11:07:58.724745 18033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:44285 every 8 connection(s)
I20260927 11:07:58.725075 17905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:07:58.727295 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 17905
I20260927 11:07:58.727372 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:07:58.727543 17791 external_mini_cluster.cc:1428] Setting key 8baac2be5eb1e5541578d7fb199ed0d0
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:07:58.736189 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:58.725875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:45223 (local address 127.17.95.193:35969)
0927 11:07:58.726046 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 11:07:58.726088 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 11:07:58.726587 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:07:58.726855 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:07:58.726859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:07:58.727553 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:07:58.727845 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:07:58.727852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.728818 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.728819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:07:58.729825 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:07:58.729828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.730215 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.730216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:07:58.730364 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:58.730897 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:07:58.730913 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:07:58.732689 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:07:58.734693 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:07:58.734699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:07:58.734700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:07:58.734906 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:07:58.735169 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:07:58.735170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:07:58.735170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:07:58.735265 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:07:58.735642 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:07:58.735647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:07:58.735759 (+ 112us) client_negotiation.cc:769] Sending connection context
0927 11:07:58.735873 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 11:07:58.736003 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260927 11:07:58.736187 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:58.725915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:35969 (local address 127.17.95.254:45223)
0927 11:07:58.725959 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:07:58.725961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:07:58.726173 (+ 212us) server_negotiation.cc:407] Connection header received
0927 11:07:58.726691 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:07:58.726692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:07:58.726708 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:07:58.726787 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:07:58.727939 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.728726 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.729914 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.730133 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.730195 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:58.732795 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:07:58.732797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:07:58.732798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:07:58.732807 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:07:58.734600 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:58.734988 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:58.734990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:58.734991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:58.735057 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:58.735347 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:58.735348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:58.735349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:58.735537 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:07:58.735631 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 11:07:58.735929 (+ 298us) server_negotiation.cc:299] Negotiation successful
0927 11:07:58.736041 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:07:58.736826 18034 heartbeater.cc:347] Connected to a master server at 127.17.95.254:45223
I20260927 11:07:58.736937 18034 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
I20260927 11:07:58.737109 18034 heartbeater.cc:511] Master 127.17.95.254:45223 requested a full tablet report, sending...
I20260927 11:07:58.737660 17843 ts_manager.cc:194] Registered new tserver with Master: 9b9d67cfb65d41fc8665387f96d32548 (127.17.95.193:44285)
I20260927 11:07:58.738294 17843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:35969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:07:58.753798 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:45223
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:07:58.835183 18039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:07:58.835397 18039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:07:58.835421 18039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:07:58.836508 18039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:07:58.836547 18039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:07:58.836575 18039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:07:58.838033 18039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:45223
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:07:58.838217 18039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:07:58.838387 18039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:07:58.843683 18044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:07:58.843719 18045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:07:58.843806 18039 server_base.cc:1034] running on GCE node
W20260927 11:07:58.843691 18047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:07:58.844054 18039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:07:58.844295 18039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:07:58.845433 18039 hybrid_clock.cc:648] HybridClock initialized: now 1790507278845420 us; error 26 us; skew 500 ppm
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:07:58.848008 18039 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:07:58.848402 18039 webserver.cc:460] Webserver started at http://127.17.95.194:34009/ using document root <none> and password file <none>
I20260927 11:07:58.848628 18039 fs_manager.cc:359] Metadata directory not provided
I20260927 11:07:58.848694 18039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:07:58.848839 18039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:07:58.849854 18039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "c91d321a00c24eeb83c0813780533c01"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "4afa6a669484408a0faf9a916ddcb9be"
server_key_iv: "002c61a2fac153202c9ffc3d8eff3605"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.850198 18039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "c91d321a00c24eeb83c0813780533c01"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "4afa6a669484408a0faf9a916ddcb9be"
server_key_iv: "002c61a2fac153202c9ffc3d8eff3605"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.851480 18039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:07:58.852155 18054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.852306 18039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:07:58.852406 18039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "c91d321a00c24eeb83c0813780533c01"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "4afa6a669484408a0faf9a916ddcb9be"
server_key_iv: "002c61a2fac153202c9ffc3d8eff3605"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.852497 18039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:07:58.865007 18039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:07:58.867225 18039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:07:58.867376 18039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:07:58.867588 18039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:07:58.867879 18039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:07:58.867929 18039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.867964 18039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:07:58.867990 18039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.873507 18039 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:33539
I20260927 11:07:58.873592 18167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:33539 every 8 connection(s)
I20260927 11:07:58.873893 18039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:07:58.878432 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18039
I20260927 11:07:58.878543 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:07:58.878701 17791 external_mini_cluster.cc:1428] Setting key 60d0404cbeae6aa02585b0bb47f69394
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:07:58.883029 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:58.874610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:52435 (local address 127.17.95.254:45223)
0927 11:07:58.874657 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:07:58.874659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:07:58.874827 (+ 168us) server_negotiation.cc:407] Connection header received
0927 11:07:58.875381 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:07:58.875382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:07:58.875391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:07:58.875454 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:07:58.876333 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.876839 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.877757 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.877949 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.878006 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:58.880239 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:07:58.880241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:07:58.880241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:07:58.880249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:07:58.881531 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:58.881996 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:58.881997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:58.881997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:58.882043 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:58.882324 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:58.882326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:58.882326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:58.882467 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:07:58.882529 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:07:58.882876 (+ 347us) server_negotiation.cc:299] Negotiation successful
0927 11:07:58.882964 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:07:58.883296 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:58.874561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:45223 (local address 127.17.95.194:52435)
0927 11:07:58.874705 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 11:07:58.874744 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:07:58.875294 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:07:58.875515 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:07:58.875518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:07:58.876056 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:07:58.876239 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:07:58.876243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.876919 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.876920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:07:58.877679 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:07:58.877681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:07:58.878017 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:07:58.878017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:07:58.878105 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:58.878588 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:07:58.878621 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:07:58.880154 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:07:58.881604 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:07:58.881610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:07:58.881613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:07:58.881918 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:07:58.882100 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:07:58.882101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:07:58.882101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:07:58.882234 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:07:58.882542 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:07:58.882548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:07:58.882707 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 11:07:58.882822 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 11:07:58.882992 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260927 11:07:58.883778 18168 heartbeater.cc:347] Connected to a master server at 127.17.95.254:45223
I20260927 11:07:58.883883 18168 heartbeater.cc:464] Registering TS with master...
I20260927 11:07:58.884050 18168 heartbeater.cc:511] Master 127.17.95.254:45223 requested a full tablet report, sending...
I20260927 11:07:58.884441 17843 ts_manager.cc:194] Registered new tserver with Master: c91d321a00c24eeb83c0813780533c01 (127.17.95.194:33539)
I20260927 11:07:58.884982 17843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:52435
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:07:58.903352 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:45223
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:07:58.982141 18173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:07:58.982334 18173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:07:58.982357 18173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:07:58.983417 18173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:07:58.983453 18173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:07:58.983479 18173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:07:58.984819 18173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:45223
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:07:58.985000 18173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:07:58.985149 18173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:07:58.989831 18181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:07:58.989848 18173 server_base.cc:1034] running on GCE node
W20260927 11:07:58.989851 18178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:07:58.989835 18179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:07:58.990248 18173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:07:58.990474 18173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:07:58.991633 18173 hybrid_clock.cc:648] HybridClock initialized: now 1790507278991611 us; error 35 us; skew 500 ppm
Sep 27 11:07:58 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:07:58.994096 18173 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:07:58.994520 18173 webserver.cc:460] Webserver started at http://127.17.95.195:33541/ using document root <none> and password file <none>
I20260927 11:07:58.994737 18173 fs_manager.cc:359] Metadata directory not provided
I20260927 11:07:58.994804 18173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:07:58.994966 18173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:07:58.996003 18173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "5abf97641bd64118b515fee70203ec3e"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "cc637e0f63a4a8b7333fa28442ebcc08"
server_key_iv: "f7d49e6c1560d6392cab87ac743cb84d"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.996349 18173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "5abf97641bd64118b515fee70203ec3e"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "cc637e0f63a4a8b7333fa28442ebcc08"
server_key_iv: "f7d49e6c1560d6392cab87ac743cb84d"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.997658 18173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:07:58.998400 18188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:58.998593 18173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:07:58.998688 18173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "5abf97641bd64118b515fee70203ec3e"
format_stamp: "Formatted at 2026-09-27 11:07:58 on dist-test-slave-6j0v"
server_key: "cc637e0f63a4a8b7333fa28442ebcc08"
server_key_iv: "f7d49e6c1560d6392cab87ac743cb84d"
server_key_version: "encryptionkey@0"
I20260927 11:07:58.998769 18173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:07:59.013001 18173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:07:59.015251 18173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:07:59.015404 18173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:07:59.015616 18173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:07:59.015921 18173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:07:59.015980 18173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:59.016033 18173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:07:59.016062 18173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:07:59.021944 18173 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:40513
I20260927 11:07:59.021991 18301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:40513 every 8 connection(s)
I20260927 11:07:59.022271 18173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:07:59.027514 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18173
I20260927 11:07:59.027626 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:07:59.027794 17791 external_mini_cluster.cc:1428] Setting key e6495425498e829d191588ae68c1e622
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507278, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:07:59.031200 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:59.023054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:59035 (local address 127.17.95.254:45223)
0927 11:07:59.023109 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 11:07:59.023111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:07:59.023290 (+ 179us) server_negotiation.cc:407] Connection header received
0927 11:07:59.023719 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:07:59.023719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:07:59.023728 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:07:59.023789 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:07:59.024604 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.025103 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.026021 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.026223 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.026288 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:59.028556 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:07:59.028558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:07:59.028558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:07:59.028565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:07:59.029949 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:59.030359 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:59.030360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:59.030360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:59.030409 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:59.030651 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:59.030651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:59.030651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:59.030745 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:07:59.030792 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:07:59.031016 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 11:07:59.031125 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 11:07:59.031474 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:59.023015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:45223 (local address 127.17.95.195:59035)
0927 11:07:59.023168 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 11:07:59.023207 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:07:59.023644 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:07:59.023841 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:07:59.023844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:07:59.024362 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:07:59.024534 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:07:59.024538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.025176 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.025177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:07:59.025941 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:07:59.025942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.026304 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.026305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:07:59.026393 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:59.026840 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:07:59.026848 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:07:59.028468 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:07:59.030043 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:07:59.030049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:07:59.030050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:07:59.030279 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:07:59.030471 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:07:59.030471 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:07:59.030471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:07:59.030594 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:07:59.030800 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:07:59.030804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:07:59.030911 (+ 107us) client_negotiation.cc:769] Sending connection context
0927 11:07:59.030997 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 11:07:59.031187 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260927 11:07:59.031968 18302 heartbeater.cc:347] Connected to a master server at 127.17.95.254:45223
I20260927 11:07:59.032080 18302 heartbeater.cc:464] Registering TS with master...
I20260927 11:07:59.032256 18302 heartbeater.cc:511] Master 127.17.95.254:45223 requested a full tablet report, sending...
I20260927 11:07:59.032680 17843 ts_manager.cc:194] Registered new tserver with Master: 5abf97641bd64118b515fee70203ec3e (127.17.95.195:40513)
I20260927 11:07:59.033018 17843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:59035
I20260927 11:07:59.039856 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:07:59.063370 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:59.055755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.17.95.193:44285)
0927 11:07:59.055921 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 11:07:59.055927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:07:59.055937 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:07:59.055963 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:07:59.055966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:07:59.055979 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:07:59.056064 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:07:59.056597 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.057304 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.058032 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.058339 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.058392 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:59.060602 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:07:59.060609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:07:59.060609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:07:59.060617 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:07:59.062250 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:59.062577 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:59.062580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:59.062580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:59.062644 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:59.062806 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:59.062806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:59.062807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:59.062935 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:07:59.063052 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 11:07:59.063142 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 11:07:59.063203 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
W20260927 11:07:59.063781 17948 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56638
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[17798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[17798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:07:59.084422 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:07:59.077771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.17.95.193:44285)
0927 11:07:59.077831 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:07:59.077833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:07:59.077926 (+ 93us) server_negotiation.cc:407] Connection header received
0927 11:07:59.077960 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:07:59.077961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:07:59.077973 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:07:59.078017 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:07:59.078510 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.078949 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.079680 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:07:59.079861 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:07:59.079910 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:07:59.081881 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:07:59.081882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:07:59.081882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:07:59.081889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:07:59.083331 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:59.083624 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:59.083625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:59.083625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:59.083678 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:07:59.083935 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:07:59.083936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:07:59.083936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:07:59.084039 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:07:59.084137 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 11:07:59.084207 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 11:07:59.084281 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:07:59.085029 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 17905
I20260927 11:07:59.089792 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18039
I20260927 11:07:59.094921 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18173
I20260927 11:07:59.098587 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 17811
2026-09-27T11:07:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1835 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[18316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[18316](info): set up 2 sockets
Sep 27 11:07:59 dist-test-slave-6j0v krb5kdc[18316](info): commencing operation
krb5kdc: starting...
W20260927 11:08:00.122563 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:00.230026 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:37227
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:37227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:00.309195 18329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:00.309428 18329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:00.309475 18329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:00.310573 18329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:00.310636 18329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:00.310659 18329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:00.310689 18329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:00.310729 18329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:00.312198 18329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:37227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:37227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:00.312502 18329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:00.312722 18329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:00.317643 18334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.317668 18337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.317668 18335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:00.317965 18329 server_base.cc:1034] running on GCE node
I20260927 11:08:00.318246 18329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:00.318567 18329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:00.319726 18329 hybrid_clock.cc:648] HybridClock initialized: now 1790507280319710 us; error 28 us; skew 500 ppm
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:00.332175 18329 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:00.332719 18329 webserver.cc:460] Webserver started at http://127.17.95.254:46161/ using document root <none> and password file <none>
I20260927 11:08:00.332993 18329 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:00.333067 18329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:00.333242 18329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:00.334399 18329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "5a95ad182761459f819b038945e2b7f8"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "53f83d433b814eaeabef979b3a526892"
server_key_iv: "9d8f07e95521eae589cbb8050a2015b1"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.334795 18329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "5a95ad182761459f819b038945e2b7f8"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "53f83d433b814eaeabef979b3a526892"
server_key_iv: "9d8f07e95521eae589cbb8050a2015b1"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.336265 18329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:00.337062 18344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.337252 18329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:00.337363 18329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "5a95ad182761459f819b038945e2b7f8"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "53f83d433b814eaeabef979b3a526892"
server_key_iv: "9d8f07e95521eae589cbb8050a2015b1"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.337436 18329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:00.346129 18329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:00.348407 18329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:00.348564 18329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:00.352497 18329 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:37227
I20260927 11:08:00.352552 18396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:37227 every 8 connection(s)
I20260927 11:08:00.352896 18329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:00.353698 18397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:00.354177 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18329
I20260927 11:08:00.354264 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:00.354419 17791 external_mini_cluster.cc:1428] Setting key 79d2176911ab648481c5bdb1107842b8
I20260927 11:08:00.356280 18397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8: Bootstrap starting.
I20260927 11:08:00.356848 18397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:00.357141 18397 log.cc:824] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:00.357869 18397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8: No bootstrap required, opened a new log
I20260927 11:08:00.360394 18397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a95ad182761459f819b038945e2b7f8" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37227 } }
I20260927 11:08:00.360594 18397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:00.360637 18397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a95ad182761459f819b038945e2b7f8, State: Initialized, Role: FOLLOWER
I20260927 11:08:00.360754 18397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a95ad182761459f819b038945e2b7f8" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37227 } }
I20260927 11:08:00.360827 18397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:00.360881 18397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:00.360930 18397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:00.361493 18397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a95ad182761459f819b038945e2b7f8" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37227 } }
I20260927 11:08:00.361618 18397 leader_election.cc:302] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a95ad182761459f819b038945e2b7f8; no voters:
I20260927 11:08:00.361792 18397 leader_election.cc:288] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:00.361851 18402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:00.362020 18402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [term 1 LEADER]: Becoming Leader. State: Replica: 5a95ad182761459f819b038945e2b7f8, State: Running, Role: LEADER
I20260927 11:08:00.362040 18397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:00.362115 18402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a95ad182761459f819b038945e2b7f8" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37227 } }
I20260927 11:08:00.362416 18402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a95ad182761459f819b038945e2b7f8. Latest consensus state: current_term: 1 leader_uuid: "5a95ad182761459f819b038945e2b7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a95ad182761459f819b038945e2b7f8" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37227 } } }
I20260927 11:08:00.362495 18402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:00.362617 18403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a95ad182761459f819b038945e2b7f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a95ad182761459f819b038945e2b7f8" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37227 } } }
I20260927 11:08:00.362733 18403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:00.363349 18408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:00.363937 18408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:00.365592 18408 catalog_manager.cc:1347] Generated new cluster ID: 0e975ace11c0490f932942bd989ca253
I20260927 11:08:00.365648 18408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:00.366101 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.354915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.17.95.254:37227)
0927 11:08:00.355135 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.355144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.355179 (+ 35us) server_negotiation.cc:407] Connection header received
0927 11:08:00.355562 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.355568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.355613 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.355759 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.356340 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.357403 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.358492 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.358873 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.358956 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.361861 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.361865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.361866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.361879 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.364074 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.364492 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.364495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.364496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.364595 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.364841 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.364842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.364842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.365126 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.365313 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.365754 (+ 441us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.365891 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260927 11:08:00.375402 18408 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:00.375972 18408 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:00.379010 18408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a95ad182761459f819b038945e2b7f8: Generated new TSK 0
I20260927 11:08:00.379204 18408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:00.410068 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37227
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:00.489667 18423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:00.489863 18423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:00.489885 18423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:00.491011 18423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:00.491047 18423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:00.491122 18423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:00.492594 18423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:00.492760 18423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:00.492921 18423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:00.497826 18429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.497875 18428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.497888 18431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:00.498181 18423 server_base.cc:1034] running on GCE node
I20260927 11:08:00.498411 18423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:00.498716 18423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:00.499892 18423 hybrid_clock.cc:648] HybridClock initialized: now 1790507280499877 us; error 29 us; skew 500 ppm
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:00.503247 18423 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:00.503830 18423 webserver.cc:460] Webserver started at http://127.17.95.193:43711/ using document root <none> and password file <none>
I20260927 11:08:00.504099 18423 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:00.504160 18423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:00.504316 18423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:00.505786 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "72a84782c8464e40b2124bdd24dd90b8"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "495556d535cf918b0f5f40d917f5dbcb"
server_key_iv: "b5312a3ad50cf7afd5c21d7027b6fe53"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.506222 18423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "72a84782c8464e40b2124bdd24dd90b8"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "495556d535cf918b0f5f40d917f5dbcb"
server_key_iv: "b5312a3ad50cf7afd5c21d7027b6fe53"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.507779 18423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260927 11:08:00.508462 18438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.508615 18423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:00.508708 18423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "72a84782c8464e40b2124bdd24dd90b8"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "495556d535cf918b0f5f40d917f5dbcb"
server_key_iv: "b5312a3ad50cf7afd5c21d7027b6fe53"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.508791 18423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:00.530706 18423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:00.532756 18423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:00.532903 18423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:00.533157 18423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:00.533452 18423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:00.533509 18423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.533550 18423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:00.533578 18423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.539371 18423 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:35249
I20260927 11:08:00.539412 18551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:35249 every 8 connection(s)
I20260927 11:08:00.539738 18423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:00.544518 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18423
I20260927 11:08:00.544618 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:00.544788 17791 external_mini_cluster.cc:1428] Setting key 637f7cff1fe5bba125756af33ddff1e1
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:00.549458 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.540498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:57559 (local address 127.17.95.254:37227)
0927 11:08:00.540599 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.540601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.540774 (+ 173us) server_negotiation.cc:407] Connection header received
0927 11:08:00.541737 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.541738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.541748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.541801 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.542343 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.542844 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.543916 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.544124 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.544179 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.546392 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.546393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.546394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.546401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.548077 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.548491 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.548492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.548492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.548566 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.548874 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.548875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.548875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.548972 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.549024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.549331 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.549396 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 11:08:00.549752 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.540456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:37227 (local address 127.17.95.193:57559)
0927 11:08:00.540649 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.540683 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.541649 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.541856 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.541859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.542091 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:00.542272 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.542276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.542939 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.542939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.543842 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.543844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.544198 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.544199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.544295 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.544815 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:00.544823 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:00.546300 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:00.548162 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:00.548168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:00.548170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:00.548406 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:00.548660 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:00.548661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:00.548662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:00.548813 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:00.549038 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:00.549044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:00.549202 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.549313 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.549488 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260927 11:08:00.550308 18552 heartbeater.cc:347] Connected to a master server at 127.17.95.254:37227
I20260927 11:08:00.550406 18552 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:00.550599 18552 heartbeater.cc:511] Master 127.17.95.254:37227 requested a full tablet report, sending...
I20260927 11:08:00.551235 18361 ts_manager.cc:194] Registered new tserver with Master: 72a84782c8464e40b2124bdd24dd90b8 (127.17.95.193:35249)
I20260927 11:08:00.551854 18361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:57559
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:00.570014 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37227
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:00.648237 18557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:00.648463 18557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:00.648499 18557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:00.649607 18557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:00.649672 18557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:00.649717 18557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:00.651177 18557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:00.651435 18557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:00.651654 18557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:00.656383 18565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:00.656432 18557 server_base.cc:1034] running on GCE node
W20260927 11:08:00.656399 18563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.656391 18562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:00.656879 18557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:00.657069 18557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:00.658229 18557 hybrid_clock.cc:648] HybridClock initialized: now 1790507280658205 us; error 37 us; skew 500 ppm
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:00.660828 18557 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:00.661227 18557 webserver.cc:460] Webserver started at http://127.17.95.194:36259/ using document root <none> and password file <none>
I20260927 11:08:00.661412 18557 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:00.661450 18557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:00.661543 18557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:00.662604 18557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "43049c63db204452aa3f216127bd7e6b"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "b7c4477eaa481b342fc39e7d845dab61"
server_key_iv: "6d7e1dd342f249ed890f3cf1277d9cca"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.662945 18557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "43049c63db204452aa3f216127bd7e6b"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "b7c4477eaa481b342fc39e7d845dab61"
server_key_iv: "6d7e1dd342f249ed890f3cf1277d9cca"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.664242 18557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:00.664884 18572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.665055 18557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:00.665149 18557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "43049c63db204452aa3f216127bd7e6b"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "b7c4477eaa481b342fc39e7d845dab61"
server_key_iv: "6d7e1dd342f249ed890f3cf1277d9cca"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.665227 18557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:00.674837 18557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:00.676594 18557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:00.676740 18557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:00.676957 18557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:00.677240 18557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:00.677294 18557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.677332 18557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:00.677361 18557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.682601 18557 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:44079
I20260927 11:08:00.682690 18685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:44079 every 8 connection(s)
I20260927 11:08:00.682932 18557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:00.684300 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18557
I20260927 11:08:00.684397 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:00.684589 17791 external_mini_cluster.cc:1428] Setting key 9dee6d548062311e05e9b457ae77814b
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
I20260927 11:08:00.692929 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.683612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:55401 (local address 127.17.95.254:37227)
0927 11:08:00.683668 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.683670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.683859 (+ 189us) server_negotiation.cc:407] Connection header received
0927 11:08:00.684967 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.684968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.684980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.685031 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.685729 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.686214 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.687421 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.687637 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.687722 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.690168 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.690169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.690170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.690175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.691559 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.692036 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.692036 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.692037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.692083 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.692354 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.692354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.692355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.692443 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.692489 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.692810 (+ 321us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.692867 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 11:08:00.693188 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.683581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:37227 (local address 127.17.95.194:55401)
0927 11:08:00.683731 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.683765 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.684883 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.685093 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.685096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.685360 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:00.685644 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.685649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.686299 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.686300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.687334 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.687336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.687730 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.687732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.687852 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.688370 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:00.688382 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:00.690079 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:00.691648 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:00.691655 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:00.691658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:00.691948 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:00.692147 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:00.692149 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:00.692149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:00.692278 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:00.692505 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:00.692511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:00.692670 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.692787 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.692953 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260927 11:08:00.693670 18686 heartbeater.cc:347] Connected to a master server at 127.17.95.254:37227
I20260927 11:08:00.693744 18686 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:00.693933 18686 heartbeater.cc:511] Master 127.17.95.254:37227 requested a full tablet report, sending...
I20260927 11:08:00.694376 18361 ts_manager.cc:194] Registered new tserver with Master: 43049c63db204452aa3f216127bd7e6b (127.17.95.194:44079)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
I20260927 11:08:00.694852 18361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:55401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:00.707834 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37227
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:00.785621 18691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:00.785840 18691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:00.785878 18691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:00.787014 18691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:00.787075 18691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:00.787160 18691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:00.788623 18691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:00.788870 18691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:00.789088 18691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:08:00.793933 18691 server_base.cc:1034] running on GCE node
W20260927 11:08:00.793903 18699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.793928 18696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:00.793908 18697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:00.794239 18691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:00.794473 18691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:00.795640 18691 hybrid_clock.cc:648] HybridClock initialized: now 1790507280795639 us; error 50 us; skew 500 ppm
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:00.797989 18691 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:00.798408 18691 webserver.cc:460] Webserver started at http://127.17.95.195:40069/ using document root <none> and password file <none>
I20260927 11:08:00.798619 18691 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:00.798703 18691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:00.798857 18691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:00.799865 18691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "362c56969ddf4921afdc48364ce63701"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "d988250422b346419f859a02c82a854b"
server_key_iv: "ac0e0e859c3a246a5abcc9f8ae15d210"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.800205 18691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "362c56969ddf4921afdc48364ce63701"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "d988250422b346419f859a02c82a854b"
server_key_iv: "ac0e0e859c3a246a5abcc9f8ae15d210"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.801549 18691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:00.802289 18706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.802459 18691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:00.802562 18691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "362c56969ddf4921afdc48364ce63701"
format_stamp: "Formatted at 2026-09-27 11:08:00 on dist-test-slave-6j0v"
server_key: "d988250422b346419f859a02c82a854b"
server_key_iv: "ac0e0e859c3a246a5abcc9f8ae15d210"
server_key_version: "encryptionkey@0"
I20260927 11:08:00.802642 18691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:00.810593 18691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:00.812912 18691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:00.813064 18691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:00.813298 18691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:00.813577 18691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:00.813628 18691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.813668 18691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:00.813695 18691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:00.819362 18691 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:45337
I20260927 11:08:00.819403 18819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:45337 every 8 connection(s)
I20260927 11:08:00.819731 18691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:00.821884 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18691
I20260927 11:08:00.821983 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:00.822140 17791 external_mini_cluster.cc:1428] Setting key f3a20f2e08996c6bb5afb028e200af61
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:00.829061 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.820428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:50497 (local address 127.17.95.254:37227)
0927 11:08:00.820536 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.820538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.820672 (+ 134us) server_negotiation.cc:407] Connection header received
0927 11:08:00.821573 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.821574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.821586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.821631 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.822450 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.823038 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.824023 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.824240 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.824323 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.826405 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.826407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.826408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.826415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.827746 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.828154 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.828155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.828156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.828213 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.828443 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.828444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.828444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.828566 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.828653 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.828901 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.828975 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 11:08:00.829232 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.820385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:37227 (local address 127.17.95.195:50497)
0927 11:08:00.820549 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.820589 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.821471 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.821705 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.821709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.822101 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:00.822360 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.822365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.823147 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.823149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.823923 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.823925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.824331 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.824332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.824458 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.824867 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:00.824874 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:00.826316 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:00.827838 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:00.827843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:00.827845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:00.828070 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:00.828274 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:00.828276 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:00.828276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:00.828380 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:00.828646 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:00.828651 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:00.828776 (+ 125us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.828870 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.829018 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260927 11:08:00.829689 18820 heartbeater.cc:347] Connected to a master server at 127.17.95.254:37227
I20260927 11:08:00.829774 18820 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:00.829962 18820 heartbeater.cc:511] Master 127.17.95.254:37227 requested a full tablet report, sending...
I20260927 11:08:00.830309 18361 ts_manager.cc:194] Registered new tserver with Master: 362c56969ddf4921afdc48364ce63701 (127.17.95.195:45337)
I20260927 11:08:00.830633 18361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:50497
I20260927 11:08:00.833786 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:08:00.840878 18400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.835552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.17.95.254:37227)
0927 11:08:00.835611 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.835613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.835628 (+ 15us) server_negotiation.cc:407] Connection header received
0927 11:08:00.835668 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.835677 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.835684 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.835735 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.836234 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.836690 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.837424 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.837596 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.837645 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.838703 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.838703 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.838704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.838709 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.839904 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.840186 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.840186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.840187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.840231 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.840411 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.840412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.840412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.840524 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.840597 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.840751 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.840822 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:08:00.842869 18361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:00.844700 18361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:00.850441 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.846726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35249 (local address 127.0.0.1:45664)
0927 11:08:00.847371 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.847378 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.847437 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.847674 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.847675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.847678 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:00.847722 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.847724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.848589 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.848590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.849341 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.849343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.850151 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.850152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.850228 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.850241 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.850321 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.850374 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":71,"threads_started":1}
I20260927 11:08:00.850453 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.846932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.17.95.193:35249)
0927 11:08:00.847130 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.847134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.847439 (+ 305us) server_negotiation.cc:407] Connection header received
0927 11:08:00.847501 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.847504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.847518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.847591 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:00.847803 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.848481 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.849531 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.850064 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.850152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.850216 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.850347 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.850384 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260927 11:08:00.851262 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.846718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:44079 (local address 127.0.0.1:58044)
0927 11:08:00.847125 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.847166 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.847261 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.847569 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.847571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.847574 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:00.847644 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.847646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.848906 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.848907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.850087 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.850089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.851004 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.851005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.851064 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.851104 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.851156 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.851197 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":48,"thread_start_us":57,"threads_started":1}
I20260927 11:08:00.851285 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.846905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.17.95.194:44079)
0927 11:08:00.847114 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.847118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.847268 (+ 150us) server_negotiation.cc:407] Connection header received
0927 11:08:00.847378 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.847380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.847395 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.847473 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:00.847726 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.848804 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.850178 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.850853 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.850915 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.850970 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.851173 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.851214 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260927 11:08:00.851302 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.846748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51582 (local address 127.17.95.195:45337)
0927 11:08:00.846889 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.846893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.847249 (+ 356us) server_negotiation.cc:407] Connection header received
0927 11:08:00.847375 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.847377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.847394 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.847478 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:00.847731 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.848794 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.850183 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.850852 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.850898 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.850944 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.851192 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.851229 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260927 11:08:00.851374 18486 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac883c7b51244fb8fd4efb64c1c845a (DEFAULT_TABLE table=test-table [id=d1d50cdb6266494bbc38dca2314a9e9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:00.851485 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.846685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:45337 (local address 127.0.0.1:51582)
0927 11:08:00.847125 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.847166 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.847261 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.847568 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.847571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.847576 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:00.847644 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.847647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.848906 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.848907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.850087 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.850089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.851003 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.851004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.851064 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.851109 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.851393 (+ 284us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.851426 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":102,"thread_start_us":51,"threads_started":1}
I20260927 11:08:00.851647 18486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac883c7b51244fb8fd4efb64c1c845a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:00.852090 18754 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac883c7b51244fb8fd4efb64c1c845a (DEFAULT_TABLE table=test-table [id=d1d50cdb6266494bbc38dca2314a9e9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:00.852310 18754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac883c7b51244fb8fd4efb64c1c845a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:00.854180 18834 tablet_bootstrap.cc:492] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8: Bootstrap starting.
I20260927 11:08:00.854583 18835 tablet_bootstrap.cc:492] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701: Bootstrap starting.
I20260927 11:08:00.855154 18620 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac883c7b51244fb8fd4efb64c1c845a (DEFAULT_TABLE table=test-table [id=d1d50cdb6266494bbc38dca2314a9e9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:00.855317 18835 tablet_bootstrap.cc:654] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:00.855321 18834 tablet_bootstrap.cc:654] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:00.855475 18620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac883c7b51244fb8fd4efb64c1c845a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:00.855712 18835 log.cc:824] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:00.855826 18834 log.cc:824] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:00.856407 18835 tablet_bootstrap.cc:492] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701: No bootstrap required, opened a new log
I20260927 11:08:00.856462 18835 ts_tablet_manager.cc:1397] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:00.856470 18834 tablet_bootstrap.cc:492] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8: No bootstrap required, opened a new log
I20260927 11:08:00.856515 18834 ts_tablet_manager.cc:1397] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:00.857803 18834 raft_consensus.cc:348] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.857923 18834 raft_consensus.cc:374] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:00.857942 18834 raft_consensus.cc:729] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a84782c8464e40b2124bdd24dd90b8, State: Initialized, Role: FOLLOWER
I20260927 11:08:00.858052 18834 consensus_queue.cc:260] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.858292 18552 heartbeater.cc:503] Master 127.17.95.254:37227 was elected leader, sending a full tablet report...
I20260927 11:08:00.858382 18835 raft_consensus.cc:348] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.858538 18835 raft_consensus.cc:374] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:00.858572 18835 raft_consensus.cc:729] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362c56969ddf4921afdc48364ce63701, State: Initialized, Role: FOLLOWER
I20260927 11:08:00.858294 18834 ts_tablet_manager.cc:1428] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:00.858698 18835 consensus_queue.cc:260] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.858985 18835 ts_tablet_manager.cc:1428] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:08:00.859167 18820 heartbeater.cc:503] Master 127.17.95.254:37227 was elected leader, sending a full tablet report...
I20260927 11:08:00.859716 18839 tablet_bootstrap.cc:492] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: Bootstrap starting.
I20260927 11:08:00.860204 18839 tablet_bootstrap.cc:654] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:00.860584 18839 log.cc:824] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:00.861784 18839 tablet_bootstrap.cc:492] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: No bootstrap required, opened a new log
I20260927 11:08:00.861846 18839 ts_tablet_manager.cc:1397] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:00.863507 18839 raft_consensus.cc:348] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.863633 18839 raft_consensus.cc:374] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:00.863674 18839 raft_consensus.cc:729] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43049c63db204452aa3f216127bd7e6b, State: Initialized, Role: FOLLOWER
I20260927 11:08:00.863812 18839 consensus_queue.cc:260] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.864073 18839 ts_tablet_manager.cc:1428] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 11:08:00.864127 18686 heartbeater.cc:503] Master 127.17.95.254:37227 was elected leader, sending a full tablet report...
I20260927 11:08:00.914201 18840 raft_consensus.cc:482] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:00.914310 18840 raft_consensus.cc:504] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.914716 18840 leader_election.cc:288] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43049c63db204452aa3f216127bd7e6b (127.17.95.194:44079), 72a84782c8464e40b2124bdd24dd90b8 (127.17.95.193:35249)
I20260927 11:08:00.918670 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.914946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:40063 (local address 127.17.95.193:35249)
0927 11:08:00.915014 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.915016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.915132 (+ 116us) server_negotiation.cc:407] Connection header received
0927 11:08:00.915194 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.915195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.915205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.915253 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:00.915398 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.916063 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.917722 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.918321 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.918391 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.918399 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.918563 (+ 164us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.918599 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 11:08:00.918680 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.914925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:43669 (local address 127.17.95.194:44079)
0927 11:08:00.914971 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.914973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.914984 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:00.915016 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.915017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.915026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.915154 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:00.915329 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.916063 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.917722 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.918308 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.918391 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.918399 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.918565 (+ 166us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.918599 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:00.918684 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.914836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:44079 (local address 127.17.95.195:43669)
0927 11:08:00.914892 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.914905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.914946 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.915162 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.915163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.915165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:00.915255 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.915257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.916147 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.916148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.917533 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.917535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.918395 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.918396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.918482 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.918484 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.918542 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.918588 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":96}
I20260927 11:08:00.918691 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.914861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35249 (local address 127.17.95.195:40063)
0927 11:08:00.915018 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:00.915027 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:00.915138 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:00.915271 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:00.915272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:00.915274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:00.915329 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.915330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.916147 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.916148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.917533 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:00.917535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.918395 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.918396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:00.918480 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.918484 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:00.918542 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 11:08:00.918588 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":66,"thread_start_us":89,"threads_started":1}
I20260927 11:08:00.919059 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac883c7b51244fb8fd4efb64c1c845a" candidate_uuid: "362c56969ddf4921afdc48364ce63701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43049c63db204452aa3f216127bd7e6b" is_pre_election: true
I20260927 11:08:00.919230 18640 raft_consensus.cc:2454] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 362c56969ddf4921afdc48364ce63701 in term 0.
I20260927 11:08:00.919363 18506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac883c7b51244fb8fd4efb64c1c845a" candidate_uuid: "362c56969ddf4921afdc48364ce63701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a84782c8464e40b2124bdd24dd90b8" is_pre_election: true
I20260927 11:08:00.919483 18709 leader_election.cc:302] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362c56969ddf4921afdc48364ce63701, 43049c63db204452aa3f216127bd7e6b; no voters:
I20260927 11:08:00.919533 18506 raft_consensus.cc:2454] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 362c56969ddf4921afdc48364ce63701 in term 0.
I20260927 11:08:00.919623 18840 raft_consensus.cc:2781] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:00.919749 18840 raft_consensus.cc:482] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:00.919766 18840 raft_consensus.cc:3037] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:00.920398 18840 raft_consensus.cc:504] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.920536 18840 leader_election.cc:288] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [CANDIDATE]: Term 1 election: Requested vote from peers 43049c63db204452aa3f216127bd7e6b (127.17.95.194:44079), 72a84782c8464e40b2124bdd24dd90b8 (127.17.95.193:35249)
I20260927 11:08:00.920680 18506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac883c7b51244fb8fd4efb64c1c845a" candidate_uuid: "362c56969ddf4921afdc48364ce63701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a84782c8464e40b2124bdd24dd90b8"
I20260927 11:08:00.920674 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac883c7b51244fb8fd4efb64c1c845a" candidate_uuid: "362c56969ddf4921afdc48364ce63701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43049c63db204452aa3f216127bd7e6b"
I20260927 11:08:00.920748 18640 raft_consensus.cc:3037] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:00.920748 18506 raft_consensus.cc:3037] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:00.921587 18506 raft_consensus.cc:2454] T 5ac883c7b51244fb8fd4efb64c1c845a P 72a84782c8464e40b2124bdd24dd90b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 362c56969ddf4921afdc48364ce63701 in term 1.
I20260927 11:08:00.921587 18640 raft_consensus.cc:2454] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 362c56969ddf4921afdc48364ce63701 in term 1.
I20260927 11:08:00.921756 18707 leader_election.cc:302] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362c56969ddf4921afdc48364ce63701, 72a84782c8464e40b2124bdd24dd90b8; no voters:
I20260927 11:08:00.921852 18840 raft_consensus.cc:2781] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:00.922029 18840 raft_consensus.cc:686] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [term 1 LEADER]: Becoming Leader. State: Replica: 362c56969ddf4921afdc48364ce63701, State: Running, Role: LEADER
I20260927 11:08:00.922123 18840 consensus_queue.cc:237] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } }
I20260927 11:08:00.922987 18361 catalog_manager.cc:5495] T 5ac883c7b51244fb8fd4efb64c1c845a P 362c56969ddf4921afdc48364ce63701 reported cstate change: term changed from 0 to 1, leader changed from <none> to 362c56969ddf4921afdc48364ce63701 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "362c56969ddf4921afdc48364ce63701" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362c56969ddf4921afdc48364ce63701" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43049c63db204452aa3f216127bd7e6b" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72a84782c8464e40b2124bdd24dd90b8" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35249 } health_report { overall_health: UNKNOWN } } }
W20260927 11:08:00.933467 18687 tablet.cc:2369] T 5ac883c7b51244fb8fd4efb64c1c845a P 43049c63db204452aa3f216127bd7e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:00.945951 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.938626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.17.95.193:35249)
0927 11:08:00.938703 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.938705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.938811 (+ 106us) server_negotiation.cc:407] Connection header received
0927 11:08:00.938840 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.938841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.938853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.938900 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.939472 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.939960 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.940759 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.940957 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.941037 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.943305 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.943313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.943314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.943320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.944941 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.945258 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.945261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.945262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.945316 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.945492 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.945493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.945493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.945622 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.945730 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.945780 (+ 50us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.945839 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:00.967896 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.960983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.17.95.193:35249)
0927 11:08:00.961046 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.961048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.961138 (+ 90us) server_negotiation.cc:407] Connection header received
0927 11:08:00.961183 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.961183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.961192 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.961248 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.961801 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.962262 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.963021 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.963279 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.963333 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.965357 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.965358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.965358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.965364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.966717 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.966977 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.966978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.966978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.967019 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.967300 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.967301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.967302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.967420 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.967543 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.967705 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.967784 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 11:08:00 dist-test-slave-6j0v krb5kdc[18316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:00.989511 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:00.982954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.17.95.193:35249)
0927 11:08:00.983030 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:00.983032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:00.983169 (+ 137us) server_negotiation.cc:407] Connection header received
0927 11:08:00.983251 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:00.983252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:00.983262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:00.983335 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:00.983876 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.984345 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.985088 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:00.985280 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:00.985362 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:00.987547 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:00.987548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:00.987549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:00.987554 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:00.988660 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.988964 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.988964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.988965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.989010 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:00.989161 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:00.989162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:00.989162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:00.989241 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:00.989307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:00.989393 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 11:08:00.989442 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260927 11:08:00.989951 18466 tablet_service.cc:2908] Rejecting scan request for tablet 5ac883c7b51244fb8fd4efb64c1c845a: Uninitialized: clean time has not yet been initialized
I20260927 11:08:00.991015 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18423
I20260927 11:08:00.996645 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18557
I20260927 11:08:01.001317 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18691
I20260927 11:08:01.005677 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18329
2026-09-27T11:08:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1907 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:01 dist-test-slave-6j0v krb5kdc[18858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:01 dist-test-slave-6j0v krb5kdc[18858](info): set up 2 sockets
Sep 27 11:08:01 dist-test-slave-6j0v krb5kdc[18858](info): commencing operation
krb5kdc: starting...
W20260927 11:08:02.018738 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:02.130021 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39713
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:39713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:02.209537 18871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:02.209769 18871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:02.209789 18871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:02.210886 18871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:02.210924 18871 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 11:08:02.210937 18871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:02.210949 18871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:02.210969 18871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:02.211020 18871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:02.212491 18871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:39713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:02.212728 18871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:02.212967 18871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:02.217720 18876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.217743 18877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.217847 18879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:02.217860 18871 server_base.cc:1034] running on GCE node
I20260927 11:08:02.218243 18871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:02.218544 18871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:02.219707 18871 hybrid_clock.cc:648] HybridClock initialized: now 1790507282219692 us; error 40 us; skew 500 ppm
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:02.222028 18871 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:02.222496 18871 webserver.cc:460] Webserver started at http://127.17.95.254:37665/ using document root <none> and password file <none>
I20260927 11:08:02.222713 18871 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:02.222779 18871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:02.222923 18871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:02.224113 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "b24cb67ffae848b7af5fcc809ef246a6"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "cc6134e8f5837c57114ea10fc2e72f99"
server_key_iv: "04e56e879422888372a00960a51d7a55"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.224498 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "b24cb67ffae848b7af5fcc809ef246a6"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "cc6134e8f5837c57114ea10fc2e72f99"
server_key_iv: "04e56e879422888372a00960a51d7a55"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.225790 18871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:08:02.226497 18886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.226671 18871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:02.226765 18871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "b24cb67ffae848b7af5fcc809ef246a6"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "cc6134e8f5837c57114ea10fc2e72f99"
server_key_iv: "04e56e879422888372a00960a51d7a55"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.226850 18871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:02.249665 18871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:02.251703 18871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:02.251883 18871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:02.256645 18871 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:39713
I20260927 11:08:02.256695 18948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:39713 every 8 connection(s)
I20260927 11:08:02.257030 18871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:02.257648 18949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:02.260026 18949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: Bootstrap starting.
I20260927 11:08:02.260619 18949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:02.260900 18949 log.cc:824] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:02.261585 18949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: No bootstrap required, opened a new log
I20260927 11:08:02.263672 18949 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24cb67ffae848b7af5fcc809ef246a6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39713 } }
I20260927 11:08:02.263805 18949 raft_consensus.cc:374] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:02.263827 18949 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24cb67ffae848b7af5fcc809ef246a6, State: Initialized, Role: FOLLOWER
I20260927 11:08:02.263995 18949 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24cb67ffae848b7af5fcc809ef246a6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39713 } }
I20260927 11:08:02.264060 18949 raft_consensus.cc:388] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:02.264101 18949 raft_consensus.cc:482] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:02.264146 18949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:02.264672 18949 raft_consensus.cc:504] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24cb67ffae848b7af5fcc809ef246a6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39713 } }
I20260927 11:08:02.264694 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18871
I20260927 11:08:02.264797 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:02.264755 18949 leader_election.cc:302] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b24cb67ffae848b7af5fcc809ef246a6; no voters:
I20260927 11:08:02.264952 18949 leader_election.cc:288] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:02.264977 17791 external_mini_cluster.cc:1428] Setting key e64b1ec2dfa9567d3b648b25e8cd05b3
I20260927 11:08:02.265003 18952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:02.265148 18952 raft_consensus.cc:686] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [term 1 LEADER]: Becoming Leader. State: Replica: b24cb67ffae848b7af5fcc809ef246a6, State: Running, Role: LEADER
I20260927 11:08:02.265218 18949 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:02.265291 18952 consensus_queue.cc:237] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24cb67ffae848b7af5fcc809ef246a6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39713 } }
I20260927 11:08:02.265630 18952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24cb67ffae848b7af5fcc809ef246a6. Latest consensus state: current_term: 1 leader_uuid: "b24cb67ffae848b7af5fcc809ef246a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24cb67ffae848b7af5fcc809ef246a6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39713 } } }
I20260927 11:08:02.265710 18952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:02.265834 18953 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b24cb67ffae848b7af5fcc809ef246a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24cb67ffae848b7af5fcc809ef246a6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39713 } } }
I20260927 11:08:02.265969 18953 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:02.266443 18961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:02.266937 18961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:02.268419 18961 catalog_manager.cc:1347] Generated new cluster ID: 6018bd5c075043329f2eafb56ed44816
I20260927 11:08:02.268469 18961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:02.276546 18961 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:02.277187 18961 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:02.277331 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.265616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.17.95.254:39713)
0927 11:08:02.266104 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.266114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.266148 (+ 34us) server_negotiation.cc:407] Connection header received
0927 11:08:02.266272 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.266282 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.266349 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.267395 (+ 1046us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.267487 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.268843 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.269880 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.270520 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.270630 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.273529 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.273533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.273533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.273544 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.275519 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.275922 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.275925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.275925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.276037 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.276371 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.276371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.276371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.276669 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.276823 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.276941 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.277083 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":68,"threads_started":1}
I20260927 11:08:02.286306 18961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: Generated new TSK 0
I20260927 11:08:02.286478 18961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:02.316042 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39713
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:02.404507 18975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:02.404733 18975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:02.404772 18975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:02.405889 18975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:02.405988 18975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:02.406028 18975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:02.406085 18975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:02.407577 18975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:39713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:02.407830 18975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:02.408058 18975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:02.413075 18981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.413167 18980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.413154 18983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:02.413293 18975 server_base.cc:1034] running on GCE node
I20260927 11:08:02.413479 18975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:02.413686 18975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:02.414830 18975 hybrid_clock.cc:648] HybridClock initialized: now 1790507282414814 us; error 28 us; skew 500 ppm
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:02.417366 18975 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:02.417820 18975 webserver.cc:460] Webserver started at http://127.17.95.193:41203/ using document root <none> and password file <none>
I20260927 11:08:02.418087 18975 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:02.418135 18975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:02.418303 18975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:02.419456 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "0f6eaef02fc0f3064792ec2102697edd"
server_key_iv: "d215b1bd746523eb2d2f749489dd205f"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.419813 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "0f6eaef02fc0f3064792ec2102697edd"
server_key_iv: "d215b1bd746523eb2d2f749489dd205f"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.421233 18975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:02.422047 18990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.422359 18975 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:02.422463 18975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "0f6eaef02fc0f3064792ec2102697edd"
server_key_iv: "d215b1bd746523eb2d2f749489dd205f"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.422538 18975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:02.443172 18975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:02.445324 18975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:02.445544 18975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:02.446126 18975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:02.446194 18975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.446234 18975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:02.446283 18975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.453680 18975 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:35455
I20260927 11:08:02.453799 19105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:35455 every 8 connection(s)
I20260927 11:08:02.454066 18975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:02.457130 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.446652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:41285 (local address 127.17.95.254:39713)
0927 11:08:02.446733 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.446735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.447059 (+ 324us) server_negotiation.cc:407] Connection header received
0927 11:08:02.447849 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.447851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.447864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.447901 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.448966 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.449828 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.451561 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.451873 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.451931 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.454817 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.454818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.454819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.454825 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.456023 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.456355 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.456356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.456356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.456401 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.456621 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.456622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.456622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.456729 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.456776 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.456988 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.457068 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 11:08:02.457329 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.446627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.17.95.193:41285)
0927 11:08:02.446882 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.446947 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.447724 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.448080 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.448086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.448527 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:02.448836 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.448844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.449957 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.449958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.451460 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.451464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.451964 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.451965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.452133 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.452883 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:02.452895 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:02.454729 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:02.456089 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.456094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.456096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.456292 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.456455 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.456456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.456456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.456565 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.456779 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:02.456783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:02.456887 (+ 104us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.456964 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.457093 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
I20260927 11:08:02.457894 19106 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39713
I20260927 11:08:02.457973 19106 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:02.458171 19106 heartbeater.cc:511] Master 127.17.95.254:39713 requested a full tablet report, sending...
I20260927 11:08:02.458717 18902 ts_manager.cc:194] Registered new tserver with Master: 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455)
I20260927 11:08:02.459583 18902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:41285
I20260927 11:08:02.461293 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 18975
I20260927 11:08:02.461391 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:02.461563 17791 external_mini_cluster.cc:1428] Setting key 254484da05ead92c6db8c60b284354f7
I20260927 11:08:02.464624 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.458872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.17.95.193:37099)
0927 11:08:02.459042 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.459054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.459121 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.459234 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.459235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.459378 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:02.459430 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.459431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.460236 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.460237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.460757 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.460759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.461059 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.461060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.461135 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.461576 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:02.461580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:02.461990 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:02.463696 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.463696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.463697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.463844 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.464061 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.464062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.464062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.464111 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.464373 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:02.464374 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:02.464416 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.464487 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.464558 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260927 11:08:02.464630 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.459056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:37099 (local address 127.17.95.254:39713)
0927 11:08:02.459130 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.459132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.459142 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:02.459162 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.459163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.459171 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.459232 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.459491 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.460150 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.460826 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.460998 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.461047 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.462060 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.462061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.462062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.462067 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.463620 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.463912 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.463913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.463913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.463980 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.464174 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.464175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.464176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.464305 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.464378 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.464507 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.464576 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:02.486583 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39713
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:02.568928 19115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:02.569106 19115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:02.569129 19115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:02.570197 19115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:02.570230 19115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:02.570250 19115 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:02.570266 19115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:02.571717 19115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:39713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:02.571899 19115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:02.572055 19115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:02.576790 19123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.576828 19121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:02.576876 19115 server_base.cc:1034] running on GCE node
W20260927 11:08:02.576830 19120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:02.577173 19115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:02.577426 19115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:02.578565 19115 hybrid_clock.cc:648] HybridClock initialized: now 1790507282578544 us; error 34 us; skew 500 ppm
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:02.580812 19115 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:02.581208 19115 webserver.cc:460] Webserver started at http://127.17.95.194:36589/ using document root <none> and password file <none>
I20260927 11:08:02.581432 19115 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:02.581494 19115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:02.581637 19115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:02.582603 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "fe7bae7619484b3daa0b1ddb60858cbb"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "53dcf6453da35010dfe72beff0baf229"
server_key_iv: "a13d40284383a5f83de70e1fbdf4b68c"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.582940 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "fe7bae7619484b3daa0b1ddb60858cbb"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "53dcf6453da35010dfe72beff0baf229"
server_key_iv: "a13d40284383a5f83de70e1fbdf4b68c"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.584355 19115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:02.585143 19130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.585322 19115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:02.585410 19115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "fe7bae7619484b3daa0b1ddb60858cbb"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "53dcf6453da35010dfe72beff0baf229"
server_key_iv: "a13d40284383a5f83de70e1fbdf4b68c"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.585487 19115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:02.606132 19115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:02.608151 19115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:02.608297 19115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:02.608824 19115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:02.608873 19115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.608894 19115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:02.608906 19115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.615882 19115 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:33399
I20260927 11:08:02.616055 19245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:33399 every 8 connection(s)
I20260927 11:08:02.616291 19115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:02.619689 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.609623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:47569 (local address 127.17.95.254:39713)
0927 11:08:02.609678 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.609680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.609903 (+ 223us) server_negotiation.cc:407] Connection header received
0927 11:08:02.610556 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.610558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.610571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.610627 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.611571 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.612374 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.613997 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.614180 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.614249 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.617034 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.617046 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.617046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.617053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.618510 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.618851 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.618852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.618852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.618901 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.619168 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.619169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.619169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.619272 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.619323 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.619572 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.619629 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 11:08:02.619835 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.609600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.17.95.194:47569)
0927 11:08:02.609783 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.609819 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.610434 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.610724 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.610728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.611159 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:02.611462 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.611470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.612479 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.612480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.613907 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.613909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.614296 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.614297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.614463 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.615042 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:02.615054 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:02.616939 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:02.618580 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.618584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.618586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.618788 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.618955 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.618956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.618956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.619049 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.619329 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:02.619333 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:02.619455 (+ 122us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.619533 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.619643 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260927 11:08:02.620349 19246 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39713
I20260927 11:08:02.620420 19246 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:02.620630 19246 heartbeater.cc:511] Master 127.17.95.254:39713 requested a full tablet report, sending...
I20260927 11:08:02.621021 18902 ts_manager.cc:194] Registered new tserver with Master: fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:02.621362 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 19115
I20260927 11:08:02.621457 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:02.621475 18902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:47569
I20260927 11:08:02.621642 17791 external_mini_cluster.cc:1428] Setting key 79f6dc6f17897a3af5cd01c5da90d803
I20260927 11:08:02.627418 19252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.621400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.17.95.194:41445)
0927 11:08:02.621683 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.621695 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.621771 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.621932 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.621934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.622166 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:02.622237 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.622239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.622885 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.622886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.623467 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.623468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.623868 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.623869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.623942 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.624387 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:02.624391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:02.624689 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:02.626252 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.626252 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.626253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.626477 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.626691 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.626692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.626692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.626754 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.627033 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:02.627034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:02.627104 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.627201 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.627316 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":80,"threads_started":1}
I20260927 11:08:02.627429 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.621464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:41445 (local address 127.17.95.254:39713)
0927 11:08:02.621502 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.621504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.621746 (+ 242us) server_negotiation.cc:407] Connection header received
0927 11:08:02.621829 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.621830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.621841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.621892 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.622327 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.622795 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.623567 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.623786 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.623867 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.624797 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.624798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.624798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.624805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.626159 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.626563 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.626564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.626565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.626622 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.626821 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.626821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.626822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.626967 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.627041 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.627242 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.627315 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:02.645633 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39713
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:02.724327 19255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:02.724573 19255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:02.724601 19255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:02.725960 19255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:02.726007 19255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:02.726050 19255 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:02.726085 19255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:02.727679 19255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:39713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:02.727912 19255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:02.728148 19255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:02.733100 19260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.733143 19263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:02.733101 19261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:02.733115 19255 server_base.cc:1034] running on GCE node
I20260927 11:08:02.733510 19255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:02.733741 19255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:02.734896 19255 hybrid_clock.cc:648] HybridClock initialized: now 1790507282734889 us; error 35 us; skew 500 ppm
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:02.737430 19255 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:02.737835 19255 webserver.cc:460] Webserver started at http://127.17.95.195:32845/ using document root <none> and password file <none>
I20260927 11:08:02.738096 19255 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:02.738165 19255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:02.738318 19255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:02.739390 19255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "b54f75a942874d61b91072b6151e5ac2"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "9a752180576c87eac788c4dc5a85af9d"
server_key_iv: "caf3523fcea5fa25bb6d8228eb9d7691"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.739728 19255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "b54f75a942874d61b91072b6151e5ac2"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "9a752180576c87eac788c4dc5a85af9d"
server_key_iv: "caf3523fcea5fa25bb6d8228eb9d7691"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.741205 19255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:02.742023 19270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.742216 19255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:02.742314 19255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "b54f75a942874d61b91072b6151e5ac2"
format_stamp: "Formatted at 2026-09-27 11:08:02 on dist-test-slave-6j0v"
server_key: "9a752180576c87eac788c4dc5a85af9d"
server_key_iv: "caf3523fcea5fa25bb6d8228eb9d7691"
server_key_version: "encryptionkey@0"
I20260927 11:08:02.742390 19255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:02.757669 19255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:02.759944 19255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:02.760097 19255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:02.760542 19255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:02.760609 19255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.760666 19255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:02.760694 19255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:02.767334 19255 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:42089
I20260927 11:08:02.767391 19385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:42089 every 8 connection(s)
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:02.767705 19255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:02.770025 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 19255
I20260927 11:08:02.770121 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:02.770251 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.761067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:60459 (local address 127.17.95.254:39713)
0927 11:08:02.761142 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.761144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.761284 (+ 140us) server_negotiation.cc:407] Connection header received
0927 11:08:02.761833 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.761835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.761846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.761928 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.762732 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.763621 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.765209 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.765529 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.765594 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.768135 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.768136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.768137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.768143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.769196 (+ 1053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.769513 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.769514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.769514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.769557 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.769717 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.769717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.769717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.769802 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.769879 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.770089 (+ 210us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.770167 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 11:08:02.770305 17791 external_mini_cluster.cc:1428] Setting key b05f0baa7d46adc0eda2eef670af85b7
I20260927 11:08:02.770455 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.761026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.17.95.195:60459)
0927 11:08:02.761161 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.761198 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.761741 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.762000 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.762003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.762358 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:02.762627 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.762633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.763710 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.763711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.765080 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.765082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.765615 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.765616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.765745 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.766328 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:02.766336 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:02.768035 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:02.769258 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.769263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.769265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.769454 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.769595 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.769595 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.769595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.769675 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.769852 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:02.769856 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:02.769978 (+ 122us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.770051 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.770212 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260927 11:08:02.770881 19386 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39713
I20260927 11:08:02.770953 19386 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:02.771153 19386 heartbeater.cc:511] Master 127.17.95.254:39713 requested a full tablet report, sending...
I20260927 11:08:02.771451 18902 ts_manager.cc:194] Registered new tserver with Master: b54f75a942874d61b91072b6151e5ac2 (127.17.95.195:42089)
I20260927 11:08:02.771901 18902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:60459
I20260927 11:08:02.777038 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.771835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:42815 (local address 127.17.95.254:39713)
0927 11:08:02.771871 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.771873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.771987 (+ 114us) server_negotiation.cc:407] Connection header received
0927 11:08:02.772108 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.772109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.772127 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.772180 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.772483 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.772983 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.773680 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.773842 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.773898 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.774780 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.774781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.774781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.774787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.776128 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.776467 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.776468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.776468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.776527 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.776693 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.776694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.776694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.776776 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.776820 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.776930 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.776985 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:02.777091 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.771767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.17.95.195:42815)
0927 11:08:02.771908 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.771917 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.772036 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.772178 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.772179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.772346 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:02.772394 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.772395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.773057 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.773058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.773600 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.773602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.773934 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.773935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.774017 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.774387 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:02.774391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:02.774699 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:02.776196 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.776197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.776198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.776375 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.776592 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:02.776592 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:02.776593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:02.776645 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:02.776822 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:02.776823 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:02.776858 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.776914 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.776992 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":53,"threads_started":1}
I20260927 11:08:02.782191 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 11:08:02 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:02.805173 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.798620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.17.95.254:39713)
0927 11:08:02.798679 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.798686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.798693 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:02.798715 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.798716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.798724 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.798774 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:02.799376 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.799862 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.800589 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.800770 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.800822 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.802849 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:02.802850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:02.802851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:02.802857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:02.804245 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.804572 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.804572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.804573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.804619 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:02.804771 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:02.804771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:02.804771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:02.804861 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:02.804930 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.805020 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.805073 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 11:08:02.806808 18902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:02.808537 18902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:02.814947 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.811039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.17.95.195:42089)
0927 11:08:02.811243 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.811248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.811408 (+ 160us) server_negotiation.cc:407] Connection header received
0927 11:08:02.811502 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.811504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.811519 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.811589 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:02.811805 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.812619 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.813996 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.814610 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.814664 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.814704 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.814855 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.814879 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":118,"threads_started":1}
I20260927 11:08:02.814966 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.810944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42089 (local address 127.0.0.1:36434)
0927 11:08:02.811338 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.811347 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.811420 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.811664 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.811667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.811669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:02.811727 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.811728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.812723 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.812724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.813908 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.813910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.814667 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.814667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.814754 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.814768 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.814847 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.814895 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":77,"threads_started":1}
I20260927 11:08:02.815578 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.811182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33399 (local address 127.0.0.1:54822)
0927 11:08:02.811650 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.811660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.811708 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.811878 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.811879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.811881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:02.811940 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.811941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.813216 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.813217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.814200 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.814203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.815353 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.815354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.815399 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.815402 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.815474 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.815513 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260927 11:08:02.815625 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.811393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.17.95.194:33399)
0927 11:08:02.811611 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.811614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.811759 (+ 145us) server_negotiation.cc:407] Connection header received
0927 11:08:02.811795 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.811796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.811809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.811881 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:02.812014 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.813104 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.814286 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.815024 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.815126 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.815193 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.815494 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.815533 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":72,"threads_started":1}
I20260927 11:08:02.815687 19320 tablet_service.cc:1467] Processing CreateTablet for tablet 6525353972b944daba8e1172d86d19cb (DEFAULT_TABLE table=test-table [id=67eec7a21c6a41f882db7233692b79b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:02.816037 19320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6525353972b944daba8e1172d86d19cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:02.816294 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.810937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35455 (local address 127.0.0.1:60764)
0927 11:08:02.811303 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:02.811343 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:02.811407 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:02.811797 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:02.811798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:02.811799 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:02.811848 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.811850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.813195 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.813196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.814209 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:02.814210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.816087 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.816088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:02.816146 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.816149 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:02.816202 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 11:08:02.816235 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":31,"thread_start_us":75,"threads_started":1}
I20260927 11:08:02.816530 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:02.811151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.17.95.193:35455)
0927 11:08:02.811587 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:02.811593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:02.811602 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:02.811642 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:02.811644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:02.811660 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:02.811732 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:02.811940 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.813105 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.815233 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:02.815917 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:02.815998 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:02.816050 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:02.816221 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 11:08:02.816267 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":87,"threads_started":1}
I20260927 11:08:02.816517 19180 tablet_service.cc:1467] Processing CreateTablet for tablet 6525353972b944daba8e1172d86d19cb (DEFAULT_TABLE table=test-table [id=67eec7a21c6a41f882db7233692b79b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:02.816841 19180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6525353972b944daba8e1172d86d19cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:02.818699 19409 tablet_bootstrap.cc:492] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Bootstrap starting.
I20260927 11:08:02.819587 19040 tablet_service.cc:1467] Processing CreateTablet for tablet 6525353972b944daba8e1172d86d19cb (DEFAULT_TABLE table=test-table [id=67eec7a21c6a41f882db7233692b79b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:02.819743 19409 tablet_bootstrap.cc:654] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:02.819880 19040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6525353972b944daba8e1172d86d19cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:02.820079 19409 log.cc:824] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:02.820210 19410 tablet_bootstrap.cc:492] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Bootstrap starting.
I20260927 11:08:02.820699 19410 tablet_bootstrap.cc:654] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:02.820977 19410 log.cc:824] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:02.821657 19409 tablet_bootstrap.cc:492] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: No bootstrap required, opened a new log
I20260927 11:08:02.821707 19409 ts_tablet_manager.cc:1397] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 11:08:02.822985 19413 tablet_bootstrap.cc:492] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Bootstrap starting.
I20260927 11:08:02.823459 19409 raft_consensus.cc:348] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:02.823593 19413 tablet_bootstrap.cc:654] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:02.823624 19409 raft_consensus.cc:374] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:02.823659 19409 raft_consensus.cc:729] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54f75a942874d61b91072b6151e5ac2, State: Initialized, Role: FOLLOWER
I20260927 11:08:02.823791 19409 consensus_queue.cc:260] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:02.823899 19413 log.cc:824] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:02.824069 19409 ts_tablet_manager.cc:1428] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:08:02.824090 19386 heartbeater.cc:503] Master 127.17.95.254:39713 was elected leader, sending a full tablet report...
I20260927 11:08:02.824476 19410 tablet_bootstrap.cc:492] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: No bootstrap required, opened a new log
I20260927 11:08:02.824544 19410 ts_tablet_manager.cc:1397] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 11:08:02.824666 19413 tablet_bootstrap.cc:492] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: No bootstrap required, opened a new log
I20260927 11:08:02.824723 19413 ts_tablet_manager.cc:1397] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:02.826468 19410 raft_consensus.cc:348] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:02.826511 19413 raft_consensus.cc:348] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:02.826642 19413 raft_consensus.cc:374] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:02.826642 19410 raft_consensus.cc:374] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:02.826673 19413 raft_consensus.cc:729] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9f05a2a8804ae3a611f2ab0b00a2f1, State: Initialized, Role: FOLLOWER
I20260927 11:08:02.826673 19410 raft_consensus.cc:729] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe7bae7619484b3daa0b1ddb60858cbb, State: Initialized, Role: FOLLOWER
I20260927 11:08:02.826783 19413 consensus_queue.cc:260] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:02.826784 19410 consensus_queue.cc:260] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:02.827032 19413 ts_tablet_manager.cc:1428] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:08:02.827032 19410 ts_tablet_manager.cc:1428] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:08:02.827109 19246 heartbeater.cc:503] Master 127.17.95.254:39713 was elected leader, sending a full tablet report...
I20260927 11:08:02.827196 19106 heartbeater.cc:503] Master 127.17.95.254:39713 was elected leader, sending a full tablet report...
W20260927 11:08:02.866901 19247 tablet.cc:2369] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:08:02.954840 19107 tablet.cc:2369] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:08:03.018334 19387 tablet.cc:2369] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:03.062805 19414 raft_consensus.cc:482] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:03.062965 19414 raft_consensus.cc:504] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:03.063365 19414 leader_election.cc:288] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455), fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:03.066610 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.063585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35455 (local address 127.17.95.195:41049)
0927 11:08:03.063668 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:03.063684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:03.063750 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:03.063892 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:03.063894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:03.063896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:03.063998 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.063999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.064716 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.064717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.065700 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.065702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.066332 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.066333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.066414 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.066418 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:03.066483 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 11:08:03.066522 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 11:08:03.066628 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.063679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:41049 (local address 127.17.95.193:35455)
0927 11:08:03.063729 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.063732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.063742 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:03.063813 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.063814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.063825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.063885 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:03.064081 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.064624 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.065784 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.066251 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.066335 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.066344 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.066503 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.066548 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 11:08:03.066699 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.063799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:50681 (local address 127.17.95.194:33399)
0927 11:08:03.063855 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.063857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.063937 (+ 80us) server_negotiation.cc:407] Connection header received
0927 11:08:03.063990 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.063991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.064000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.064062 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:03.064244 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.064782 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.065968 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.066385 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.066457 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.066480 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.066593 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.066627 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:03.066757 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.063702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33399 (local address 127.17.95.195:50681)
0927 11:08:03.063872 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:03.063881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:03.063928 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:03.064081 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:03.064082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:03.064083 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:03.064153 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.064155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.064865 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.064866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.065886 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.065888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.066451 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.066452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.066521 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.066524 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:03.066578 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 11:08:03.066607 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260927 11:08:03.067008 19060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6525353972b944daba8e1172d86d19cb" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" is_pre_election: true
I20260927 11:08:03.067046 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6525353972b944daba8e1172d86d19cb" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" is_pre_election: true
I20260927 11:08:03.067173 19060 raft_consensus.cc:2454] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 0.
I20260927 11:08:03.067245 19200 raft_consensus.cc:2454] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 0.
I20260927 11:08:03.067461 19274 leader_election.cc:302] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54f75a942874d61b91072b6151e5ac2, fe7bae7619484b3daa0b1ddb60858cbb; no voters:
I20260927 11:08:03.067591 19414 raft_consensus.cc:2781] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:03.067646 19414 raft_consensus.cc:482] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:03.067662 19414 raft_consensus.cc:3037] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.068578 19414 raft_consensus.cc:504] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:03.068696 19414 leader_election.cc:288] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455), fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:03.068853 19060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6525353972b944daba8e1172d86d19cb" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1"
I20260927 11:08:03.068917 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6525353972b944daba8e1172d86d19cb" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7bae7619484b3daa0b1ddb60858cbb"
I20260927 11:08:03.068933 19060 raft_consensus.cc:3037] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.069002 19200 raft_consensus.cc:3037] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.069620 19060 raft_consensus.cc:2454] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 1.
I20260927 11:08:03.069780 19200 raft_consensus.cc:2454] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 1.
I20260927 11:08:03.069780 19274 leader_election.cc:302] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9f05a2a8804ae3a611f2ab0b00a2f1, b54f75a942874d61b91072b6151e5ac2; no voters:
I20260927 11:08:03.069865 19414 raft_consensus.cc:2781] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:03.070009 19414 raft_consensus.cc:686] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 1 LEADER]: Becoming Leader. State: Replica: b54f75a942874d61b91072b6151e5ac2, State: Running, Role: LEADER
I20260927 11:08:03.070104 19414 consensus_queue.cc:237] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } }
I20260927 11:08:03.070909 18903 catalog_manager.cc:5495] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54f75a942874d61b91072b6151e5ac2 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:03.096614 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.094175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36446 (local address 127.17.95.195:42089)
0927 11:08:03.094236 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.094238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.094247 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:03.094310 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.094311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.094328 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.094391 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:03.094538 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.094992 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.095801 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.096015 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.096086 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.096276 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:03.096377 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:03.096475 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.096499 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.096516 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:08:03.098361 19060 raft_consensus.cc:1261] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Refusing update from remote peer b54f75a942874d61b91072b6151e5ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:03.098381 19200 raft_consensus.cc:1261] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Refusing update from remote peer b54f75a942874d61b91072b6151e5ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:03.098702 19414 consensus_queue.cc:1035] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:03.098839 19419 consensus_queue.cc:1035] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:03.100852 19422 mvcc.cc:205] Tried to move back new op lower bound from 7333917831567872000 to 7333917831455735808. Current Snapshot: MvccSnapshot[applied={T|T < 7333917831567872000}]
I20260927 11:08:03.101739 19423 mvcc.cc:205] Tried to move back new op lower bound from 7333917831567872000 to 7333917831455735808. Current Snapshot: MvccSnapshot[applied={T|T < 7333917831567872000 or (T in {7333917831567872000})}]
I20260927 11:08:03.101850 19424 mvcc.cc:205] Tried to move back new op lower bound from 7333917831567872000 to 7333917831455735808. Current Snapshot: MvccSnapshot[applied={T|T < 7333917831567872000 or (T in {7333917831567872000})}]
I20260927 11:08:03.104239 18902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:03.104324 18902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:03.105072 18902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: Sending DeleteTablet for 3 replicas of tablet 6525353972b944daba8e1172d86d19cb
I20260927 11:08:03.105414 19320 tablet_service.cc:1514] Processing DeleteTablet for tablet 6525353972b944daba8e1172d86d19cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:03 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:36434
I20260927 11:08:03.105401 19180 tablet_service.cc:1514] Processing DeleteTablet for tablet 6525353972b944daba8e1172d86d19cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:03 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:54822
I20260927 11:08:03.105682 19040 tablet_service.cc:1514] Processing DeleteTablet for tablet 6525353972b944daba8e1172d86d19cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:03 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:60764
I20260927 11:08:03.105840 19432 tablet_replica.cc:330] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: stopping tablet replica
I20260927 11:08:03.105937 19432 raft_consensus.cc:2229] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:03.105888 18902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:03.106019 18902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:03.106046 19432 raft_consensus.cc:2258] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:03.106176 19434 tablet_replica.cc:330] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: stopping tablet replica
I20260927 11:08:03.106312 19434 raft_consensus.cc:2229] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:03.106449 19434 raft_consensus.cc:2258] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:03.106472 19432 ts_tablet_manager.cc:1905] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:03.106911 19434 ts_tablet_manager.cc:1905] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:03.107148 19320 tablet_service.cc:1467] Processing CreateTablet for tablet 666101c66ab64b99b35ddf718c6662b2 (DEFAULT_TABLE table=test-table [id=92099df586104e8eb119c6c8c360cb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:03.107278 19320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666101c66ab64b99b35ddf718c6662b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:03.108354 19432 ts_tablet_manager.cc:1918] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:03.108422 19432 log.cc:1196] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/6525353972b944daba8e1172d86d19cb
I20260927 11:08:03.108667 19409 tablet_bootstrap.cc:492] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Bootstrap starting.
I20260927 11:08:03.108922 19434 ts_tablet_manager.cc:1918] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:03.108994 19040 tablet_service.cc:1467] Processing CreateTablet for tablet 666101c66ab64b99b35ddf718c6662b2 (DEFAULT_TABLE table=test-table [id=92099df586104e8eb119c6c8c360cb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:03.109098 19409 tablet_bootstrap.cc:654] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:03.109086 19434 log.cc:1196] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/6525353972b944daba8e1172d86d19cb
I20260927 11:08:03.109270 19180 tablet_service.cc:1467] Processing CreateTablet for tablet 666101c66ab64b99b35ddf718c6662b2 (DEFAULT_TABLE table=test-table [id=92099df586104e8eb119c6c8c360cb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:03.109369 19180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666101c66ab64b99b35ddf718c6662b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:03.109493 19432 ts_tablet_manager.cc:1939] T 6525353972b944daba8e1172d86d19cb P fe7bae7619484b3daa0b1ddb60858cbb: Deleting consensus metadata
I20260927 11:08:03.109705 19434 ts_tablet_manager.cc:1939] T 6525353972b944daba8e1172d86d19cb P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Deleting consensus metadata
I20260927 11:08:03.110106 19409 tablet_bootstrap.cc:492] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: No bootstrap required, opened a new log
I20260927 11:08:03.110157 19409 ts_tablet_manager.cc:1397] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:03.110289 19409 raft_consensus.cc:348] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.110355 19409 raft_consensus.cc:374] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:03.110381 19409 raft_consensus.cc:729] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54f75a942874d61b91072b6151e5ac2, State: Initialized, Role: FOLLOWER
I20260927 11:08:03.110430 19409 consensus_queue.cc:260] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.110526 19409 ts_tablet_manager.cc:1428] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:03.110577 19410 tablet_bootstrap.cc:492] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Bootstrap starting.
I20260927 11:08:03.109174 19040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666101c66ab64b99b35ddf718c6662b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:03.111270 18890 catalog_manager.cc:4841] TS 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455): tablet 6525353972b944daba8e1172d86d19cb (table test-table [id=67eec7a21c6a41f882db7233692b79b9]) successfully deleted
I20260927 11:08:03.111363 18890 catalog_manager.cc:4841] TS fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399): tablet 6525353972b944daba8e1172d86d19cb (table test-table [id=67eec7a21c6a41f882db7233692b79b9]) successfully deleted
I20260927 11:08:03.111375 19410 tablet_bootstrap.cc:654] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:03.111994 19410 tablet_bootstrap.cc:492] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: No bootstrap required, opened a new log
I20260927 11:08:03.112046 19410 ts_tablet_manager.cc:1397] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:03.112165 19410 raft_consensus.cc:348] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.112229 19410 raft_consensus.cc:374] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:03.112248 19410 raft_consensus.cc:729] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe7bae7619484b3daa0b1ddb60858cbb, State: Initialized, Role: FOLLOWER
I20260927 11:08:03.112290 19410 consensus_queue.cc:260] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.112387 19410 ts_tablet_manager.cc:1428] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:03.112636 19413 tablet_bootstrap.cc:492] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Bootstrap starting.
I20260927 11:08:03.112780 19433 tablet_replica.cc:330] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: stopping tablet replica
I20260927 11:08:03.112884 19433 raft_consensus.cc:2229] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:03.112973 19433 raft_consensus.cc:2258] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:03.113049 19413 tablet_bootstrap.cc:654] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:03.113343 19433 ts_tablet_manager.cc:1905] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:03.114151 19413 tablet_bootstrap.cc:492] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: No bootstrap required, opened a new log
I20260927 11:08:03.114210 19413 ts_tablet_manager.cc:1397] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:08:03.114341 19413 raft_consensus.cc:348] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.114418 19413 raft_consensus.cc:374] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:03.114440 19413 raft_consensus.cc:729] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9f05a2a8804ae3a611f2ab0b00a2f1, State: Initialized, Role: FOLLOWER
I20260927 11:08:03.114485 19413 consensus_queue.cc:260] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.114784 19413 ts_tablet_manager.cc:1428] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:03.114957 19433 ts_tablet_manager.cc:1918] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:03.115027 19433 log.cc:1196] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/6525353972b944daba8e1172d86d19cb
I20260927 11:08:03.115332 19433 ts_tablet_manager.cc:1939] T 6525353972b944daba8e1172d86d19cb P b54f75a942874d61b91072b6151e5ac2: Deleting consensus metadata
I20260927 11:08:03.115736 18888 catalog_manager.cc:4841] TS b54f75a942874d61b91072b6151e5ac2 (127.17.95.195:42089): tablet 6525353972b944daba8e1172d86d19cb (table test-table [id=67eec7a21c6a41f882db7233692b79b9]) successfully deleted
I20260927 11:08:03.122040 19428 raft_consensus.cc:482] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:03.122102 19428 raft_consensus.cc:504] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.122211 19428 leader_election.cc:288] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455), fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:03.122380 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666101c66ab64b99b35ddf718c6662b2" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" is_pre_election: true
I20260927 11:08:03.122463 19200 raft_consensus.cc:2454] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 0.
I20260927 11:08:03.122447 19060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666101c66ab64b99b35ddf718c6662b2" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" is_pre_election: true
I20260927 11:08:03.122512 19060 raft_consensus.cc:2454] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 0.
I20260927 11:08:03.122597 19274 leader_election.cc:302] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54f75a942874d61b91072b6151e5ac2, fe7bae7619484b3daa0b1ddb60858cbb; no voters:
I20260927 11:08:03.122674 19428 raft_consensus.cc:2781] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:03.122713 19428 raft_consensus.cc:482] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:03.122761 19428 raft_consensus.cc:3037] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.123471 19428 raft_consensus.cc:504] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.123585 19428 leader_election.cc:288] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455), fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:03.123749 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666101c66ab64b99b35ddf718c6662b2" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7bae7619484b3daa0b1ddb60858cbb"
I20260927 11:08:03.123740 19060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666101c66ab64b99b35ddf718c6662b2" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1"
I20260927 11:08:03.123804 19060 raft_consensus.cc:3037] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.123804 19200 raft_consensus.cc:3037] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.124460 19060 raft_consensus.cc:2454] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 1.
I20260927 11:08:03.124460 19200 raft_consensus.cc:2454] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 1.
I20260927 11:08:03.124620 19274 leader_election.cc:302] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9f05a2a8804ae3a611f2ab0b00a2f1, b54f75a942874d61b91072b6151e5ac2; no voters:
I20260927 11:08:03.124743 19428 raft_consensus.cc:2781] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:03.124827 19428 raft_consensus.cc:686] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 1 LEADER]: Becoming Leader. State: Replica: b54f75a942874d61b91072b6151e5ac2, State: Running, Role: LEADER
I20260927 11:08:03.124871 19428 consensus_queue.cc:237] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.125377 18902 catalog_manager.cc:5495] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54f75a942874d61b91072b6151e5ac2 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } health_report { overall_health: HEALTHY } } }
Sep 27 11:08:03 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:03.155226 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.147750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39713 (local address 127.0.0.1:50414)
0927 11:08:03.147894 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:03.147911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:03.147967 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:03.148108 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:03.148109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:03.148386 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:03.148430 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.148432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.149375 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.149376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.149972 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.149974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.150425 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.150425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.150462 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.151006 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:03.151010 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:03.152513 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:03.154137 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:03.154142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:03.154144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:03.154332 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:03.154527 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:03.154527 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:03.154527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:03.154581 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:03.154830 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:03.154835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:03.154872 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 11:08:03.154920 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 11:08:03.155011 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260927 11:08:03.155228 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.147811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.17.95.254:39713)
0927 11:08:03.147864 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.147866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.147973 (+ 107us) server_negotiation.cc:407] Connection header received
0927 11:08:03.148013 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.148014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.148025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.148097 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:03.148506 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.149302 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.150053 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.150348 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.150412 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.152593 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:03.152595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:03.152595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:03.152601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:03.154043 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:03.154403 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:03.154404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:03.154405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:03.154463 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:03.154640 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:03.154641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:03.154641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:03.154770 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:03.154836 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.154979 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.155098 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 11:08:03.157238 18902 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:50414:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 11:08:03.157445 18902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:03.158864 19320 tablet_service.cc:1467] Processing CreateTablet for tablet c8161f0fc359419ea93ffea9022774dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f7492e6f44bf41cc985037b451d17395]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:03.158876 19180 tablet_service.cc:1467] Processing CreateTablet for tablet c8161f0fc359419ea93ffea9022774dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f7492e6f44bf41cc985037b451d17395]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:03.158877 19040 tablet_service.cc:1467] Processing CreateTablet for tablet c8161f0fc359419ea93ffea9022774dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f7492e6f44bf41cc985037b451d17395]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:03.159003 19180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8161f0fc359419ea93ffea9022774dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:03.159008 19040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8161f0fc359419ea93ffea9022774dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:03.158996 19320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8161f0fc359419ea93ffea9022774dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:03.160352 19409 tablet_bootstrap.cc:492] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: Bootstrap starting.
I20260927 11:08:03.160374 19413 tablet_bootstrap.cc:492] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Bootstrap starting.
I20260927 11:08:03.160375 19410 tablet_bootstrap.cc:492] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: Bootstrap starting.
I20260927 11:08:03.160742 19409 tablet_bootstrap.cc:654] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:03.160750 19410 tablet_bootstrap.cc:654] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:03.160753 19413 tablet_bootstrap.cc:654] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:03.161823 19413 tablet_bootstrap.cc:492] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: No bootstrap required, opened a new log
I20260927 11:08:03.161860 19410 tablet_bootstrap.cc:492] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: No bootstrap required, opened a new log
I20260927 11:08:03.161878 19413 ts_tablet_manager.cc:1397] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:08:03.161895 19410 ts_tablet_manager.cc:1397] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:08:03.162014 19413 raft_consensus.cc:348] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.162017 19410 raft_consensus.cc:348] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.162082 19413 raft_consensus.cc:374] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:03.162086 19410 raft_consensus.cc:374] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:03.162106 19413 raft_consensus.cc:729] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9f05a2a8804ae3a611f2ab0b00a2f1, State: Initialized, Role: FOLLOWER
I20260927 11:08:03.162106 19410 raft_consensus.cc:729] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe7bae7619484b3daa0b1ddb60858cbb, State: Initialized, Role: FOLLOWER
I20260927 11:08:03.162148 19410 consensus_queue.cc:260] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.162148 19413 consensus_queue.cc:260] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.162231 19413 ts_tablet_manager.cc:1428] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:03.162235 19410 ts_tablet_manager.cc:1428] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:03.162358 19417 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.162350 19416 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.162426 19416 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.162426 19417 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.162880 19409 tablet_bootstrap.cc:492] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: No bootstrap required, opened a new log
I20260927 11:08:03.162927 19409 ts_tablet_manager.cc:1397] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 11:08:03.163017 19409 raft_consensus.cc:348] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.163126 19409 raft_consensus.cc:374] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:03.163146 19409 raft_consensus.cc:729] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54f75a942874d61b91072b6151e5ac2, State: Initialized, Role: FOLLOWER
I20260927 11:08:03.163180 19409 consensus_queue.cc:260] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.163261 19409 ts_tablet_manager.cc:1428] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:03.163271 19428 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.163345 19428 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.243681 19428 raft_consensus.cc:482] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:03.243783 19428 raft_consensus.cc:504] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.243950 19428 leader_election.cc:288] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455), fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:03.244202 19060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8161f0fc359419ea93ffea9022774dc" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" is_pre_election: true
I20260927 11:08:03.244261 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8161f0fc359419ea93ffea9022774dc" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" is_pre_election: true
I20260927 11:08:03.244346 19060 raft_consensus.cc:2454] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 0.
I20260927 11:08:03.244382 19200 raft_consensus.cc:2454] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 0.
I20260927 11:08:03.244581 19274 leader_election.cc:302] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b54f75a942874d61b91072b6151e5ac2, fe7bae7619484b3daa0b1ddb60858cbb; no voters:
I20260927 11:08:03.244688 19428 raft_consensus.cc:2781] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:03.244761 19428 raft_consensus.cc:482] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:03.244822 19428 raft_consensus.cc:3037] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.245854 19428 raft_consensus.cc:504] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.246022 19428 leader_election.cc:288] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455), fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399)
I20260927 11:08:03.246250 19060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8161f0fc359419ea93ffea9022774dc" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1"
I20260927 11:08:03.246335 19060 raft_consensus.cc:3037] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.246276 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8161f0fc359419ea93ffea9022774dc" candidate_uuid: "b54f75a942874d61b91072b6151e5ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe7bae7619484b3daa0b1ddb60858cbb"
I20260927 11:08:03.246371 19200 raft_consensus.cc:3037] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:03.247243 19060 raft_consensus.cc:2454] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 1.
I20260927 11:08:03.247243 19200 raft_consensus.cc:2454] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54f75a942874d61b91072b6151e5ac2 in term 1.
I20260927 11:08:03.247447 19274 leader_election.cc:302] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9f05a2a8804ae3a611f2ab0b00a2f1, b54f75a942874d61b91072b6151e5ac2; no voters:
I20260927 11:08:03.247545 19428 raft_consensus.cc:2781] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:03.247624 19428 raft_consensus.cc:686] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [term 1 LEADER]: Becoming Leader. State: Replica: b54f75a942874d61b91072b6151e5ac2, State: Running, Role: LEADER
I20260927 11:08:03.247684 19428 consensus_queue.cc:237] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } }
I20260927 11:08:03.247963 19419 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: TxnStatusTablet state changed. Reason: New leader b54f75a942874d61b91072b6151e5ac2. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.248064 19419 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:03.248286 19449 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 11:08:03.248413 19449 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 11:08:03.248567 18901 catalog_manager.cc:5495] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54f75a942874d61b91072b6151e5ac2 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } health_report { overall_health: HEALTHY } } }
I20260927 11:08:03.263672 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.261049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36458 (local address 127.17.95.195:42089)
0927 11:08:03.261103 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.261105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.261114 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:03.261167 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.261168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.261179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.261240 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:03.261321 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.262103 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.262871 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.263192 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.263258 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.263468 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:03.263500 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:03.263539 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.263598 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.263620 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 11:08:03.263718 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.260965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42089 (local address 127.0.0.1:36458)
0927 11:08:03.261039 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:03.261051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:03.261103 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:03.261244 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:03.261244 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:03.261245 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:03.261286 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.261287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.262173 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.262174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.262786 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.262788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.263295 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.263296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.263363 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.263379 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:03.263540 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:03.263545 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:03.263596 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 11:08:03.263641 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 11:08:03.264614 19060 raft_consensus.cc:1261] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Refusing update from remote peer b54f75a942874d61b91072b6151e5ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:03.264647 19200 raft_consensus.cc:1261] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Refusing update from remote peer b54f75a942874d61b91072b6151e5ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:03.264810 19428 consensus_queue.cc:1035] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:03.264933 19419 consensus_queue.cc:1035] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:03.265394 19417 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: TxnStatusTablet state changed. Reason: New leader b54f75a942874d61b91072b6151e5ac2. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.265516 19417 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.265823 19416 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: TxnStatusTablet state changed. Reason: New leader b54f75a942874d61b91072b6151e5ac2. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.265926 19416 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.266628 19428 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.266731 19428 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:03.266978 19417 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.267055 19417 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P 2f9f05a2a8804ae3a611f2ab0b00a2f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.267320 19414 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.267390 19414 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P b54f75a942874d61b91072b6151e5ac2: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:03.267709 19416 tablet_replica.cc:439] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b54f75a942874d61b91072b6151e5ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 } } peers { permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 } } peers { permanent_uuid: "b54f75a942874d61b91072b6151e5ac2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42089 } } }
I20260927 11:08:03.267793 19416 tablet_replica.cc:442] T c8161f0fc359419ea93ffea9022774dc P fe7bae7619484b3daa0b1ddb60858cbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:03.273041 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.270434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:60267 (local address 127.17.95.195:42089)
0927 11:08:03.270466 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.270468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.270590 (+ 122us) server_negotiation.cc:407] Connection header received
0927 11:08:03.270637 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.270638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.270644 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.270681 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:03.270783 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.271375 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.272451 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.272631 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.272686 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.272853 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:03.272876 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:03.272912 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.272977 (+ 65us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.272995 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":14}
I20260927 11:08:03.273128 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.270407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42089 (local address 127.17.95.195:60267)
0927 11:08:03.270514 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:03.270526 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:03.270584 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:03.270692 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:03.270693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:03.270694 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:03.270733 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.270735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.271459 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.271460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.272370 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:03.272372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.272703 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.272704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:03.272761 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.272776 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:03.272931 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:03.272932 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 11:08:03.272978 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 11:08:03.273028 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":3}
I20260927 11:08:03.275197 19200 raft_consensus.cc:1261] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Refusing update from remote peer b54f75a942874d61b91072b6151e5ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:03.275403 19060 raft_consensus.cc:1261] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Refusing update from remote peer b54f75a942874d61b91072b6151e5ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:03.275574 19419 consensus_queue.cc:1035] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe7bae7619484b3daa0b1ddb60858cbb" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:03.275688 19428 consensus_queue.cc:1035] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9f05a2a8804ae3a611f2ab0b00a2f1" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:03.286221 18901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:03.286325 18901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:03.287052 18901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b24cb67ffae848b7af5fcc809ef246a6: Sending DeleteTablet for 3 replicas of tablet 666101c66ab64b99b35ddf718c6662b2
I20260927 11:08:03.287343 19320 tablet_service.cc:1514] Processing DeleteTablet for tablet 666101c66ab64b99b35ddf718c6662b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:03 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:36434
I20260927 11:08:03.287343 19180 tablet_service.cc:1514] Processing DeleteTablet for tablet 666101c66ab64b99b35ddf718c6662b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:03 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:54822
I20260927 11:08:03.287433 19432 tablet_replica.cc:330] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: stopping tablet replica
I20260927 11:08:03.287431 19433 tablet_replica.cc:330] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: stopping tablet replica
I20260927 11:08:03.287484 19432 raft_consensus.cc:2229] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:03.287488 19433 raft_consensus.cc:2229] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:03.287516 19432 raft_consensus.cc:2258] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:03.287537 19433 raft_consensus.cc:2258] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:03.287835 19433 ts_tablet_manager.cc:1905] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:03.287926 19040 tablet_service.cc:1514] Processing DeleteTablet for tablet 666101c66ab64b99b35ddf718c6662b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:03 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:60764
I20260927 11:08:03.288012 19434 tablet_replica.cc:330] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: stopping tablet replica
I20260927 11:08:03.288056 19434 raft_consensus.cc:2229] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:03.288089 19434 raft_consensus.cc:2258] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:03.288399 19434 ts_tablet_manager.cc:1905] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:03.288410 19432 ts_tablet_manager.cc:1905] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:03.289345 19433 ts_tablet_manager.cc:1918] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:03.289394 19433 log.cc:1196] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/666101c66ab64b99b35ddf718c6662b2
I20260927 11:08:03.289613 19433 ts_tablet_manager.cc:1939] T 666101c66ab64b99b35ddf718c6662b2 P b54f75a942874d61b91072b6151e5ac2: Deleting consensus metadata
I20260927 11:08:03.289888 19432 ts_tablet_manager.cc:1918] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:03.289888 19434 ts_tablet_manager.cc:1918] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:03.289944 19432 log.cc:1196] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/666101c66ab64b99b35ddf718c6662b2
I20260927 11:08:03.289944 19434 log.cc:1196] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/666101c66ab64b99b35ddf718c6662b2
I20260927 11:08:03.290093 18888 catalog_manager.cc:4841] TS b54f75a942874d61b91072b6151e5ac2 (127.17.95.195:42089): tablet 666101c66ab64b99b35ddf718c6662b2 (table test-table [id=92099df586104e8eb119c6c8c360cb0c]) successfully deleted
I20260927 11:08:03.290156 19432 ts_tablet_manager.cc:1939] T 666101c66ab64b99b35ddf718c6662b2 P fe7bae7619484b3daa0b1ddb60858cbb: Deleting consensus metadata
I20260927 11:08:03.290156 19434 ts_tablet_manager.cc:1939] T 666101c66ab64b99b35ddf718c6662b2 P 2f9f05a2a8804ae3a611f2ab0b00a2f1: Deleting consensus metadata
I20260927 11:08:03.290742 18890 catalog_manager.cc:4841] TS fe7bae7619484b3daa0b1ddb60858cbb (127.17.95.194:33399): tablet 666101c66ab64b99b35ddf718c6662b2 (table test-table [id=92099df586104e8eb119c6c8c360cb0c]) successfully deleted
I20260927 11:08:03.290813 18890 catalog_manager.cc:4841] TS 2f9f05a2a8804ae3a611f2ab0b00a2f1 (127.17.95.193:35455): tablet 666101c66ab64b99b35ddf718c6662b2 (table test-table [id=92099df586104e8eb119c6c8c360cb0c]) successfully deleted
Sep 27 11:08:03 dist-test-slave-6j0v krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:03.298794 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.288037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.17.95.193:35455)
0927 11:08:03.288074 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.288076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.290513 (+ 2437us) server_negotiation.cc:407] Connection header received
0927 11:08:03.290539 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.290540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.290551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.290611 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:03.291330 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.291812 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.292526 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.292733 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.292816 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.295866 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:03.295874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:03.295874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:03.295881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:03.297651 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:03.297997 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:03.298000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:03.298001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:03.298069 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:03.298252 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:03.298253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:03.298253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:03.298404 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:03.298558 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.298615 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.298689 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 11:08:03.299144 19085 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768
I20260927 11:08:03.305819 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:03.299967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.17.95.254:39713)
0927 11:08:03.300014 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:03.300016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:03.300176 (+ 160us) server_negotiation.cc:407] Connection header received
0927 11:08:03.300246 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:03.300246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:03.300256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:03.300307 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:03.300773 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.301241 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.301921 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:03.302129 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:03.302223 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:03.303405 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:03.303406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:03.303407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:03.303413 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:03.304706 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:03.305060 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:03.305061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:03.305061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:03.305118 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:03.305327 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:03.305328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:03.305328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:03.305439 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:03.305555 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:03.305650 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 11:08:03.305722 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260927 11:08:03.306052 18901 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50428
I20260927 11:08:03.306741 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18975
I20260927 11:08:03.312443 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 19115
I20260927 11:08:03.318596 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 19255
I20260927 11:08:03.324378 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 18871
2026-09-27T11:08:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2318 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:03 dist-test-slave-6j0v krb5kdc[19464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:03 dist-test-slave-6j0v krb5kdc[19464](info): set up 2 sockets
Sep 27 11:08:03 dist-test-slave-6j0v krb5kdc[19464](info): commencing operation
krb5kdc: starting...
W20260927 11:08:04.339255 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:04.450775 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34361
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:04.533047 19477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:04.533245 19477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:04.533268 19477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:04.534330 19477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:04.534369 19477 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 11:08:04.534384 19477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:04.534399 19477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:04.534413 19477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:04.534426 19477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:04.535902 19477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34361
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:04.536095 19477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:04.536257 19477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:04.540910 19485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:04.540946 19482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:04.540913 19483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:04.541160 19477 server_base.cc:1034] running on GCE node
I20260927 11:08:04.541401 19477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:04.541698 19477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:04.542848 19477 hybrid_clock.cc:648] HybridClock initialized: now 1790507284542838 us; error 30 us; skew 500 ppm
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:04.545528 19477 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:04.546018 19477 webserver.cc:460] Webserver started at http://127.17.95.254:43181/ using document root <none> and password file <none>
I20260927 11:08:04.546234 19477 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:04.546273 19477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:04.546352 19477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:04.547340 19477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "181de01f62da45e5bb7b59f932a356d4"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "abca8181bce6190f13e4b1a4d066702c"
server_key_iv: "2661fbe5222d487547777ccc09d3c4a2"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.547628 19477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "181de01f62da45e5bb7b59f932a356d4"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "abca8181bce6190f13e4b1a4d066702c"
server_key_iv: "2661fbe5222d487547777ccc09d3c4a2"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.548802 19477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:04.549417 19492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.549638 19477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.549724 19477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "181de01f62da45e5bb7b59f932a356d4"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "abca8181bce6190f13e4b1a4d066702c"
server_key_iv: "2661fbe5222d487547777ccc09d3c4a2"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.549794 19477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:04.563927 19477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:04.566360 19477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:04.566514 19477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:04.571336 19477 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:34361
I20260927 11:08:04.571430 19554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:34361 every 8 connection(s)
I20260927 11:08:04.571739 19477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:04.572392 19555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:04.574739 19555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Bootstrap starting.
I20260927 11:08:04.575205 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 19477
I20260927 11:08:04.575274 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:04.575464 19555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:04.575474 17791 external_mini_cluster.cc:1428] Setting key 81e0abab96cc332539ce9b8efa4c5a06
I20260927 11:08:04.575810 19555 log.cc:824] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:04.576530 19555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: No bootstrap required, opened a new log
I20260927 11:08:04.578372 19555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181de01f62da45e5bb7b59f932a356d4" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34361 } }
I20260927 11:08:04.578501 19555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:04.578523 19555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181de01f62da45e5bb7b59f932a356d4, State: Initialized, Role: FOLLOWER
I20260927 11:08:04.578665 19555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181de01f62da45e5bb7b59f932a356d4" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34361 } }
I20260927 11:08:04.578743 19555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:04.578781 19555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:04.578827 19555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:04.579492 19555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181de01f62da45e5bb7b59f932a356d4" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34361 } }
I20260927 11:08:04.579619 19555 leader_election.cc:302] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 181de01f62da45e5bb7b59f932a356d4; no voters:
I20260927 11:08:04.579803 19555 leader_election.cc:288] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:04.579854 19560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:04.580005 19560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [term 1 LEADER]: Becoming Leader. State: Replica: 181de01f62da45e5bb7b59f932a356d4, State: Running, Role: LEADER
I20260927 11:08:04.580055 19555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:04.580090 19560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181de01f62da45e5bb7b59f932a356d4" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34361 } }
I20260927 11:08:04.580404 19560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 181de01f62da45e5bb7b59f932a356d4. Latest consensus state: current_term: 1 leader_uuid: "181de01f62da45e5bb7b59f932a356d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181de01f62da45e5bb7b59f932a356d4" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34361 } } }
I20260927 11:08:04.580539 19560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:04.580405 19561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "181de01f62da45e5bb7b59f932a356d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181de01f62da45e5bb7b59f932a356d4" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34361 } } }
I20260927 11:08:04.580725 19561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:04.580761 19567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:04.581296 19567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:04.582964 19567 catalog_manager.cc:1347] Generated new cluster ID: 673a35eb928b4321adadb27f29251863
I20260927 11:08:04.583179 19567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:04.587873 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.576056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.17.95.254:34361)
0927 11:08:04.576231 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:04.576240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:04.576273 (+ 33us) server_negotiation.cc:407] Connection header received
0927 11:08:04.576750 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:04.576756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:04.576802 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:04.576958 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:04.577560 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.578476 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.579696 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.580353 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.580858 (+ 505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.584042 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:04.584046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:04.584046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:04.584058 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:04.586043 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.586521 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.586524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.586524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.586620 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.586826 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.586827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.586827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.587129 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:04.587275 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:04.587527 (+ 252us) server_negotiation.cc:299] Negotiation successful
0927 11:08:04.587671 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260927 11:08:04.599828 19567 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:04.600322 19567 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:04.604928 19567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Generated new TSK 0
I20260927 11:08:04.605106 19567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:04.633277 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34361
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:04.715451 19581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:04.715631 19581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:04.715653 19581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:04.716794 19581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:04.716835 19581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:04.716856 19581 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 11:08:04.716890 19581 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:04.716924 19581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:04.718396 19581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:34361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:04.718639 19581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:04.718860 19581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:04.723733 19587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:04.723786 19586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:04.723786 19581 server_base.cc:1034] running on GCE node
W20260927 11:08:04.723738 19589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:04.724153 19581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:04.724395 19581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:04.725533 19581 hybrid_clock.cc:648] HybridClock initialized: now 1790507284725520 us; error 27 us; skew 500 ppm
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:04.728034 19581 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:04.728451 19581 webserver.cc:460] Webserver started at http://127.17.95.193:39421/ using document root <none> and password file <none>
I20260927 11:08:04.728685 19581 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:04.728750 19581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:04.728894 19581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:04.729863 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "b329ba9c33764ba184b6b1542560dc5a"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "9f5e57e90d9eeb5d1d809beebbdb965b"
server_key_iv: "fedf78036cc11147660183d702ba9c5e"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.730202 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "b329ba9c33764ba184b6b1542560dc5a"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "9f5e57e90d9eeb5d1d809beebbdb965b"
server_key_iv: "fedf78036cc11147660183d702ba9c5e"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.731514 19581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:04.732245 19596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.732434 19581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:04.732530 19581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "b329ba9c33764ba184b6b1542560dc5a"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "9f5e57e90d9eeb5d1d809beebbdb965b"
server_key_iv: "fedf78036cc11147660183d702ba9c5e"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.732605 19581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:04.750205 19581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:04.752457 19581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:04.752604 19581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:04.753109 19581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:04.753172 19581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.753216 19581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:04.753242 19581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:04.760277 19581 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:44119
I20260927 11:08:04.760367 19711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:44119 every 8 connection(s)
I20260927 11:08:04.760654 19581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:04.763239 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.753886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:35341 (local address 127.17.95.254:34361)
0927 11:08:04.753944 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:04.753947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:04.753960 (+ 13us) server_negotiation.cc:407] Connection header received
0927 11:08:04.754435 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:04.754437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:04.754450 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:04.754519 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:04.755319 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.756180 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.757203 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.757413 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.757505 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.760194 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:04.760195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:04.760196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:04.760203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:04.761780 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.762192 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.762194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.762194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.762249 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.762522 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.762523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.762523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.762656 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:04.762726 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:04.763033 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 11:08:04.763149 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 11:08:04.763401 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.753640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.17.95.193:35341)
0927 11:08:04.753829 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:04.753874 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:04.754330 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:04.754568 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:04.754572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:04.754886 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:04.755211 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.755218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.756291 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.756291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.757099 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.757101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.757527 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.757528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.757656 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.758188 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:04.758199 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:04.760089 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:04.761877 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.761883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.761885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.762103 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.762330 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.762330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.762331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.762436 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.762728 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:04.762735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:04.762864 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 11:08:04.763009 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 11:08:04.763180 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260927 11:08:04.763912 19712 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34361
I20260927 11:08:04.763991 19712 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:04.764174 19712 heartbeater.cc:511] Master 127.17.95.254:34361 requested a full tablet report, sending...
I20260927 11:08:04.764693 19509 ts_manager.cc:194] Registered new tserver with Master: b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119)
I20260927 11:08:04.765333 19509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:35341
I20260927 11:08:04.768167 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 19581
I20260927 11:08:04.768268 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:04.768451 17791 external_mini_cluster.cc:1428] Setting key b5747dc327b4c17737aab1c491f1bc71
I20260927 11:08:04.770455 19718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.764854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.17.95.193:41855)
0927 11:08:04.764970 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:04.764978 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:04.765049 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:04.765176 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:04.765176 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:04.765346 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:04.765429 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.765431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.766214 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.766215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.766752 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.766753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.767149 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.767150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.767243 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.767562 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:04.767566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:04.767811 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:04.769449 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.769450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.769450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.769610 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.769810 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.769810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.769811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.769888 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.770120 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:04.770121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:04.770189 (+ 68us) client_negotiation.cc:769] Sending connection context
0927 11:08:04.770271 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:08:04.770365 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260927 11:08:04.770450 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.764894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:41855 (local address 127.17.95.254:34361)
0927 11:08:04.764931 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:04.764933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:04.765036 (+ 103us) server_negotiation.cc:407] Connection header received
0927 11:08:04.765105 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:04.765106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:04.765115 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:04.765171 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:04.765496 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.766139 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.766839 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.767042 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.767148 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.767905 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:04.767906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:04.767907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:04.767913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:04.769362 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.769680 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.769681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.769682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.769731 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.769949 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.769950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.769950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.770055 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:04.770128 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:04.770297 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 11:08:04.770365 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:04.792104 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34361
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:04.870396 19721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:04.870582 19721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:04.870604 19721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:04.871731 19721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:04.871771 19721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:04.871790 19721 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 11:08:04.871802 19721 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:04.871842 19721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:04.873270 19721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:34361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:04.873482 19721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:04.873708 19721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:04.878396 19727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:04.878404 19726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:04.878566 19721 server_base.cc:1034] running on GCE node
W20260927 11:08:04.878410 19729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:04.878872 19721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:04.879161 19721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:04.880302 19721 hybrid_clock.cc:648] HybridClock initialized: now 1790507284880283 us; error 39 us; skew 500 ppm
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:04.882913 19721 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:04.883343 19721 webserver.cc:460] Webserver started at http://127.17.95.194:39227/ using document root <none> and password file <none>
I20260927 11:08:04.883534 19721 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:04.883575 19721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:04.883661 19721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:04.884610 19721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "90a20c835c1f40c782a31a27e6629cc7"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "0aa1053b0866158842a9a1e44e6c2c8d"
server_key_iv: "a5848798edccd1e35c1e24d543e47310"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.884899 19721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "90a20c835c1f40c782a31a27e6629cc7"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "0aa1053b0866158842a9a1e44e6c2c8d"
server_key_iv: "a5848798edccd1e35c1e24d543e47310"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.886165 19721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:04.886793 19736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.886957 19721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:04.887034 19721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "90a20c835c1f40c782a31a27e6629cc7"
format_stamp: "Formatted at 2026-09-27 11:08:04 on dist-test-slave-6j0v"
server_key: "0aa1053b0866158842a9a1e44e6c2c8d"
server_key_iv: "a5848798edccd1e35c1e24d543e47310"
server_key_version: "encryptionkey@0"
I20260927 11:08:04.887149 19721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:04.898813 19721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:04.900470 19721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:04.900568 19721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:04.900944 19721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:04.900975 19721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.900995 19721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:04.901048 19721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:04.907745 19721 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:33521
I20260927 11:08:04.907819 19851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:33521 every 8 connection(s)
I20260927 11:08:04.908145 19721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:08:04 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:04.915336 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.901480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:53583 (local address 127.17.95.254:34361)
0927 11:08:04.901532 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:04.901534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:04.901713 (+ 179us) server_negotiation.cc:407] Connection header received
0927 11:08:04.902163 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:04.902165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:04.902178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:04.902244 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:04.903100 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.903971 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.905744 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.906060 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.906133 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.912782 (+ 6649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:04.912783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:04.912784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:04.912791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:04.913999 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.914404 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.914404 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.914405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.914454 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.914740 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.914740 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.914741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.914839 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:04.914888 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:04.915212 (+ 324us) server_negotiation.cc:299] Negotiation successful
0927 11:08:04.915272 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:04.915541 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.901464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.17.95.194:53583)
0927 11:08:04.901616 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:04.901646 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:04.902056 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:04.902310 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:04.902314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:04.902688 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:04.902965 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.902973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.904088 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.904089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.905633 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.905637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.906149 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.906150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.906256 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.911256 (+ 5000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:04.911264 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:04.912692 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:04.914097 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.914103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.914104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.914336 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.914559 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.914560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.914560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.914679 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.914900 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:04.914904 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:04.915027 (+ 123us) client_negotiation.cc:769] Sending connection context
0927 11:08:04.915184 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 11:08:04.915313 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260927 11:08:04.916077 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 19721
I20260927 11:08:04.916131 19852 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34361
I20260927 11:08:04.916188 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:04.916219 19852 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:04.916390 17791 external_mini_cluster.cc:1428] Setting key 208b2f11224c3fa268838bce644606a7
I20260927 11:08:04.916414 19852 heartbeater.cc:511] Master 127.17.95.254:34361 requested a full tablet report, sending...
I20260927 11:08:04.916751 19509 ts_manager.cc:194] Registered new tserver with Master: 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521)
I20260927 11:08:04.917275 19509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:53583
I20260927 11:08:04.923135 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.917166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:44477 (local address 127.17.95.254:34361)
0927 11:08:04.917270 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:04.917272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:04.917300 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:08:04.917387 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:04.917388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:04.917398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:04.917426 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:04.917837 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.918552 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.919375 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.919592 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.919684 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.920577 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:04.920578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:04.920578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:04.920585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:04.921943 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.922301 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.922302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.922303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.922353 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:04.922584 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:04.922585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:04.922585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:04.922732 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:04.922813 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:04.922963 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 11:08:04.923037 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 11:08:04.923182 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:04.917120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.17.95.194:44477)
0927 11:08:04.917243 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:04.917253 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:04.917312 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:04.917482 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:04.917483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:04.917690 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:04.917749 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.917751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.918696 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.918697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.919301 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:04.919303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:04.919687 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:04.919688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:04.919759 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:04.920206 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:04.920210 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:04.920472 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:04.922040 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.922041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.922042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.922219 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.922425 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:04.922426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:04.922427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:04.922483 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:04.922805 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:04.922806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:04.922850 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 11:08:04.922938 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 11:08:04.923033 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:04.942276 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34361
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:05.023250 19861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:05.023458 19861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:05.023480 19861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:05.024681 19861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:05.024726 19861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:05.024746 19861 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 11:08:05.024758 19861 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:05.024803 19861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:05.026259 19861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:34361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:05.026499 19861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:05.026718 19861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:05.031528 19867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:05.031561 19866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:05.031653 19861 server_base.cc:1034] running on GCE node
W20260927 11:08:05.031533 19869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:05.031900 19861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:05.032141 19861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:05.033303 19861 hybrid_clock.cc:648] HybridClock initialized: now 1790507285033288 us; error 28 us; skew 500 ppm
Sep 27 11:08:05 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507285, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:05.036129 19861 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:05.036616 19861 webserver.cc:460] Webserver started at http://127.17.95.195:33311/ using document root <none> and password file <none>
I20260927 11:08:05.036829 19861 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:05.036872 19861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:05.037038 19861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:05.038048 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "8273ae9fc7044cd3894e1392421753a1"
format_stamp: "Formatted at 2026-09-27 11:08:05 on dist-test-slave-6j0v"
server_key: "b959f3547ba554628eb6503040e81fc6"
server_key_iv: "d9337f32d46bfe99c43be2d6e5a2637a"
server_key_version: "encryptionkey@0"
I20260927 11:08:05.038398 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "8273ae9fc7044cd3894e1392421753a1"
format_stamp: "Formatted at 2026-09-27 11:08:05 on dist-test-slave-6j0v"
server_key: "b959f3547ba554628eb6503040e81fc6"
server_key_iv: "d9337f32d46bfe99c43be2d6e5a2637a"
server_key_version: "encryptionkey@0"
I20260927 11:08:05.039749 19861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260927 11:08:05.040446 19876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:05.040642 19861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:05.040726 19861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "8273ae9fc7044cd3894e1392421753a1"
format_stamp: "Formatted at 2026-09-27 11:08:05 on dist-test-slave-6j0v"
server_key: "b959f3547ba554628eb6503040e81fc6"
server_key_iv: "d9337f32d46bfe99c43be2d6e5a2637a"
server_key_version: "encryptionkey@0"
I20260927 11:08:05.040797 19861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:05.052224 19861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:05.054625 19861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:05.054770 19861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:05.055387 19861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:05.055428 19861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:05.055449 19861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:05.055467 19861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:05.063382 19861 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:42685
I20260927 11:08:05.064007 19861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:05.064785 19991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:42685 every 8 connection(s)
Sep 27 11:08:05 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507285, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:05.066915 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 19861
I20260927 11:08:05.067039 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:05.067310 17791 external_mini_cluster.cc:1428] Setting key 9373d97e518f7e48a49c7a1a6ac235ec
I20260927 11:08:05.071029 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.055828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:57603 (local address 127.17.95.254:34361)
0927 11:08:05.055899 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.055902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.056179 (+ 277us) server_negotiation.cc:407] Connection header received
0927 11:08:05.056926 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.056927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.056952 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.057045 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:05.057908 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.058750 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.061716 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.063205 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.063317 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.066781 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:05.066783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:05.066783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:05.066793 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:05.069003 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.069880 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.069881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.069882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.069962 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.070259 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.070260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.070260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.070399 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:05.070467 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.070884 (+ 417us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.070968 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 11:08:05.071296 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.055827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.17.95.195:57603)
0927 11:08:05.056031 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.056080 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.056796 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.057125 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.057130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.057509 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:05.057803 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.057810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.058848 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.058849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.061597 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.061601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.063332 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.063333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.063477 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.064133 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:05.064143 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:05.066668 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:05.069124 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:05.069131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:05.069133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:05.069637 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:05.070026 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:05.070027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:05.070027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:05.070194 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:05.070484 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:05.070489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:05.070720 (+ 231us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.070859 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.071013 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260927 11:08:05.071827 19992 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34361
I20260927 11:08:05.071906 19992 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:05.072093 19992 heartbeater.cc:511] Master 127.17.95.254:34361 requested a full tablet report, sending...
I20260927 11:08:05.072513 19508 ts_manager.cc:194] Registered new tserver with Master: 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685)
I20260927 11:08:05.073045 19508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:57603
I20260927 11:08:05.080502 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:08:05.080754 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.073043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.17.95.195:42951)
0927 11:08:05.073315 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.073329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.073482 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.073699 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.073700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.073927 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:05.074006 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.074008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.075153 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.075155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.075995 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.075997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.076628 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.076629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.076708 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.077189 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:05.077193 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:05.077611 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:05.079528 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:05.079529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:05.079529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:05.079770 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:05.080000 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:05.080001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:05.080001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:05.080060 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:05.080332 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:05.080333 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:05.080388 (+ 55us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.080498 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.080677 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":81,"threads_started":1}
I20260927 11:08:05.081108 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.073105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:42951 (local address 127.17.95.254:34361)
0927 11:08:05.073144 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.073146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.073381 (+ 235us) server_negotiation.cc:407] Connection header received
0927 11:08:05.073618 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.073619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.073630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.073695 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:05.074077 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.075029 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.076082 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.076515 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.076627 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.077700 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:05.077702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:05.077702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:05.077709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:05.079421 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.079871 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.079872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.079872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.079936 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.080130 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.080131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.080131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.080275 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:05.080335 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.080523 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.080665 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 27 11:08:05 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 11:08:05 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:05.111037 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.101770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.17.95.254:34361)
0927 11:08:05.101812 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.101815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.101823 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:05.101849 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.101850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.101861 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.101915 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:05.102674 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.103575 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.104612 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.104963 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.105056 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.108093 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:05.108094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:05.108095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:05.108103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:05.109803 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.110226 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.110228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.110228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.110290 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.110492 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.110493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.110493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.110630 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:05.110732 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.110833 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.110912 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:05.112900 19508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:05.114956 19508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:05.121196 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.117321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.17.95.195:42685)
0927 11:08:05.117686 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.117689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.117697 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:05.117730 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.117732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.117745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.117836 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.118009 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.118655 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120073 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.120791 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120877 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.120930 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.121062 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.121098 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":76,"threads_started":1}
I20260927 11:08:05.121191 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.117176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.0.0.1:43192)
0927 11:08:05.117620 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.117629 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.117678 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.117839 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.117840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.117842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.117887 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.117889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.118735 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.118736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.119988 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.119990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.120878 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.120948 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.120960 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.121037 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.121086 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":23,"thread_start_us":66,"threads_started":1}
I20260927 11:08:05.121342 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.117483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.17.95.193:44119)
0927 11:08:05.117655 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.117660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.117827 (+ 167us) server_negotiation.cc:407] Connection header received
0927 11:08:05.117863 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.117865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.117880 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.117946 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.118122 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.119059 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120082 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.120816 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120879 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.120930 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.121214 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.121251 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260927 11:08:05.121462 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.117462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.17.95.194:33521)
0927 11:08:05.117607 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.117613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.117622 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:05.117662 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.117665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.117679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.117757 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.117972 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.119021 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120311 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.120994 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.121235 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.121288 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.121312 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.121349 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260927 11:08:05.121491 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.117176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33521 (local address 127.0.0.1:32960)
0927 11:08:05.117322 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.117358 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.117444 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.117765 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.117768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.117770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.117891 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.117893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.119125 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.119127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.120238 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.120240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.121075 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.121076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.121123 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.121125 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.121151 (+ 26us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.121179 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":24,"thread_start_us":41,"threads_started":1}
I20260927 11:08:05.121706 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.117196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44119 (local address 127.0.0.1:33494)
0927 11:08:05.117738 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.117745 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.117779 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.118019 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.118020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.118022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.118054 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.118056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.119153 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.119154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.119984 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.119986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.120961 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.120962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.121038 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.121043 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.121084 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.121119 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260927 11:08:05.122248 19926 tablet_service.cc:1467] Processing CreateTablet for tablet de6671e293d84dc2b95afa5fc2e7741d (DEFAULT_TABLE table=test-table [id=e22530aeefce4b198aa3c06780ae7142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:05.122295 19646 tablet_service.cc:1467] Processing CreateTablet for tablet de6671e293d84dc2b95afa5fc2e7741d (DEFAULT_TABLE table=test-table [id=e22530aeefce4b198aa3c06780ae7142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:05.122310 19786 tablet_service.cc:1467] Processing CreateTablet for tablet de6671e293d84dc2b95afa5fc2e7741d (DEFAULT_TABLE table=test-table [id=e22530aeefce4b198aa3c06780ae7142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:05.122596 19926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6671e293d84dc2b95afa5fc2e7741d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.122596 19646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6671e293d84dc2b95afa5fc2e7741d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.122665 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6671e293d84dc2b95afa5fc2e7741d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.125464 20015 tablet_bootstrap.cc:492] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Bootstrap starting.
I20260927 11:08:05.126217 20015 tablet_bootstrap.cc:654] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.126233 20017 tablet_bootstrap.cc:492] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Bootstrap starting.
I20260927 11:08:05.126307 20016 tablet_bootstrap.cc:492] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Bootstrap starting.
I20260927 11:08:05.126646 20015 log.cc:824] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:05.126845 20017 tablet_bootstrap.cc:654] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.127262 20016 tablet_bootstrap.cc:654] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.127375 20017 log.cc:824] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:05.127462 20015 tablet_bootstrap.cc:492] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: No bootstrap required, opened a new log
I20260927 11:08:05.127526 20015 ts_tablet_manager.cc:1397] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:05.127673 20016 log.cc:824] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:05.128319 20016 tablet_bootstrap.cc:492] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: No bootstrap required, opened a new log
I20260927 11:08:05.128400 20016 ts_tablet_manager.cc:1397] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:05.129341 20015 raft_consensus.cc:348] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.129477 20015 raft_consensus.cc:374] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.129498 20015 raft_consensus.cc:729] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.129643 20015 consensus_queue.cc:260] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.129904 20015 ts_tablet_manager.cc:1428] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:05.129920 19852 heartbeater.cc:503] Master 127.17.95.254:34361 was elected leader, sending a full tablet report...
I20260927 11:08:05.130280 20016 raft_consensus.cc:348] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.130436 20016 raft_consensus.cc:374] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.130477 20016 raft_consensus.cc:729] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.130501 20017 tablet_bootstrap.cc:492] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: No bootstrap required, opened a new log
I20260927 11:08:05.130565 20017 ts_tablet_manager.cc:1397] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 11:08:05.130584 20016 consensus_queue.cc:260] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.130818 20016 ts_tablet_manager.cc:1428] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:08:05.130954 19992 heartbeater.cc:503] Master 127.17.95.254:34361 was elected leader, sending a full tablet report...
I20260927 11:08:05.132426 20017 raft_consensus.cc:348] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.132547 20017 raft_consensus.cc:374] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.132570 20017 raft_consensus.cc:729] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b329ba9c33764ba184b6b1542560dc5a, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.132668 20017 consensus_queue.cc:260] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.132874 20017 ts_tablet_manager.cc:1428] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:05.132949 19712 heartbeater.cc:503] Master 127.17.95.254:34361 was elected leader, sending a full tablet report...
W20260927 11:08:05.158742 19853 tablet.cc:2369] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:05.192902 20021 raft_consensus.cc:482] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:05.193086 20021 raft_consensus.cc:504] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.193526 20021 leader_election.cc:288] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685), b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119)
I20260927 11:08:05.197381 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.193716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.17.95.194:55427)
0927 11:08:05.193756 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.193768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.193807 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.193978 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.193979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.193981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.194054 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.194055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.195120 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.195121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.196334 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.196337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.197071 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.197072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.197184 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.197187 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.197241 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.197282 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":49}
I20260927 11:08:05.197392 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.193796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:55427 (local address 127.17.95.195:42685)
0927 11:08:05.193849 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.193851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.193859 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:05.193905 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.193905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.193913 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.193971 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.194151 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.195015 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.196448 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.196987 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.197057 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.197067 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.197279 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.197323 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:08:05.197391 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.193715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44119 (local address 127.17.95.194:47845)
0927 11:08:05.193932 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.193940 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.193994 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.194120 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.194121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.194123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.194175 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.194176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.195142 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.195143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.196334 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.196337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.197071 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.197072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.197173 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.197177 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.197246 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.197282 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":84,"thread_start_us":125,"threads_started":1}
I20260927 11:08:05.197392 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.193789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:47845 (local address 127.17.95.193:44119)
0927 11:08:05.193836 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.193838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.194014 (+ 176us) server_negotiation.cc:407] Connection header received
0927 11:08:05.194042 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.194042 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.194056 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.194099 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.194248 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.195037 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.196454 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.196973 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.197057 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.197067 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.197279 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.197325 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:05.197844 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6671e293d84dc2b95afa5fc2e7741d" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a" is_pre_election: true
I20260927 11:08:05.197849 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6671e293d84dc2b95afa5fc2e7741d" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8273ae9fc7044cd3894e1392421753a1" is_pre_election: true
I20260927 11:08:05.197996 19941 raft_consensus.cc:2454] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 0.
I20260927 11:08:05.197996 19666 raft_consensus.cc:2454] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 0.
I20260927 11:08:05.198256 19737 leader_election.cc:302] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a20c835c1f40c782a31a27e6629cc7, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:05.198431 20021 raft_consensus.cc:2781] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:05.198503 20021 raft_consensus.cc:482] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:05.198532 20021 raft_consensus.cc:3037] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:05.199595 20021 raft_consensus.cc:504] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.199759 20021 leader_election.cc:288] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 election: Requested vote from peers 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685), b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119)
I20260927 11:08:05.199930 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6671e293d84dc2b95afa5fc2e7741d" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8273ae9fc7044cd3894e1392421753a1"
I20260927 11:08:05.199957 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6671e293d84dc2b95afa5fc2e7741d" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a"
I20260927 11:08:05.200018 19941 raft_consensus.cc:3037] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:05.200037 19666 raft_consensus.cc:3037] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:05.200742 19666 raft_consensus.cc:2454] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 1.
I20260927 11:08:05.200851 19941 raft_consensus.cc:2454] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 1.
I20260927 11:08:05.200995 19737 leader_election.cc:302] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a20c835c1f40c782a31a27e6629cc7, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:05.201102 20021 raft_consensus.cc:2781] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:05.201272 20021 raft_consensus.cc:686] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 1 LEADER]: Becoming Leader. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Running, Role: LEADER
I20260927 11:08:05.201390 20021 consensus_queue.cc:237] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } }
I20260927 11:08:05.202210 19508 catalog_manager.cc:5495] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194). New cstate: current_term: 1 leader_uuid: "90a20c835c1f40c782a31a27e6629cc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } health_report { overall_health: UNKNOWN } } }
Sep 27 11:08:05 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507284, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:05.234694 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.227765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.17.95.254:34361)
0927 11:08:05.227828 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.227830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.227939 (+ 109us) server_negotiation.cc:407] Connection header received
0927 11:08:05.228007 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.228008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.228018 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.228083 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:05.228516 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.228959 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.229587 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.229749 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.229824 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.232019 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:05.232020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:05.232021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:05.232027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:05.233400 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.233851 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.233853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.233853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.233910 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:05.234101 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:05.234102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:05.234102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:05.234249 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:05.234337 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.234488 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.234570 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:08:05.234824 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.227702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34361 (local address 127.0.0.1:40942)
0927 11:08:05.227867 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.227878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.227951 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.228086 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.228087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.228385 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:05.228424 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.228427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.229033 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.229034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.229511 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.229512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.229824 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.229825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.229875 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.230471 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:05.230476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:05.231896 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:05.233531 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:05.233538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:05.233541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:05.233741 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:05.233976 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:05.233977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:05.233978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:05.234038 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:05.234327 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:05.234334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:05.234379 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.234463 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.234626 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":59,"thread_start_us":83,"threads_started":1}
I20260927 11:08:05.236863 19508 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:40942:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 11:08:05.237120 19508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:05.238603 19646 tablet_service.cc:1467] Processing CreateTablet for tablet 2919cc309c1444f7a286b697190c68fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39e2b9bc8a214e5788b08737d2a15c70]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:05.238601 19926 tablet_service.cc:1467] Processing CreateTablet for tablet 2919cc309c1444f7a286b697190c68fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39e2b9bc8a214e5788b08737d2a15c70]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:05.238593 19786 tablet_service.cc:1467] Processing CreateTablet for tablet 2919cc309c1444f7a286b697190c68fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39e2b9bc8a214e5788b08737d2a15c70]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:05.238750 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2919cc309c1444f7a286b697190c68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.238750 19926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2919cc309c1444f7a286b697190c68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.238770 19646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2919cc309c1444f7a286b697190c68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.240271 20016 tablet_bootstrap.cc:492] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: Bootstrap starting.
I20260927 11:08:05.240271 20015 tablet_bootstrap.cc:492] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: Bootstrap starting.
I20260927 11:08:05.240300 20017 tablet_bootstrap.cc:492] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: Bootstrap starting.
I20260927 11:08:05.240756 20015 tablet_bootstrap.cc:654] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.240841 20016 tablet_bootstrap.cc:654] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.240881 20017 tablet_bootstrap.cc:654] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.241506 20017 tablet_bootstrap.cc:492] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: No bootstrap required, opened a new log
I20260927 11:08:05.241540 20015 tablet_bootstrap.cc:492] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: No bootstrap required, opened a new log
I20260927 11:08:05.241559 20017 ts_tablet_manager.cc:1397] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:05.241580 20015 ts_tablet_manager.cc:1397] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:05.241688 20016 tablet_bootstrap.cc:492] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: No bootstrap required, opened a new log
I20260927 11:08:05.241721 20016 ts_tablet_manager.cc:1397] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:05.241712 20015 raft_consensus.cc:348] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.241775 20015 raft_consensus.cc:374] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.241708 20017 raft_consensus.cc:348] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.241796 20015 raft_consensus.cc:729] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.241799 20017 raft_consensus.cc:374] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.241822 20017 raft_consensus.cc:729] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b329ba9c33764ba184b6b1542560dc5a, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.241848 20015 consensus_queue.cc:260] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.241827 20016 raft_consensus.cc:348] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.241860 20017 consensus_queue.cc:260] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.241909 20016 raft_consensus.cc:374] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.241932 20016 raft_consensus.cc:729] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.241945 20015 ts_tablet_manager.cc:1428] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:05.241958 20017 ts_tablet_manager.cc:1428] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:05.241972 20016 consensus_queue.cc:260] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.241993 20021 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.242065 20016 ts_tablet_manager.cc:1428] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:05.242069 20021 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:05.242077 20023 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.242141 20023 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:05.242329 20022 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.242446 20022 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260927 11:08:05.261700 19713 tablet.cc:2369] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:08:05.315191 19993 tablet.cc:2369] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:05.588577 20021 consensus_queue.cc:1035] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:05.591266 20038 consensus_queue.cc:1035] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:05.616545 20022 raft_consensus.cc:482] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:05.616627 20022 raft_consensus.cc:504] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.616989 20022 leader_election.cc:288] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119), 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521)
I20260927 11:08:05.620915 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.617224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:56565 (local address 127.17.95.194:33521)
0927 11:08:05.617277 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.617279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.617333 (+ 54us) server_negotiation.cc:407] Connection header received
0927 11:08:05.617409 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.617410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.617419 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.617481 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.617692 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.618403 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.619961 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.620513 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.620593 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.620601 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.620814 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.620848 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:05.620915 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.617190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:38635 (local address 127.17.95.193:44119)
0927 11:08:05.617245 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.617247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.617371 (+ 124us) server_negotiation.cc:407] Connection header received
0927 11:08:05.617420 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.617421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.617429 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.617481 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:05.617688 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.618403 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.619959 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.620513 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.620593 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.620601 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.620815 (+ 214us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.620849 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 11:08:05.620947 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.617137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44119 (local address 127.17.95.195:38635)
0927 11:08:05.617305 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.617314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.617366 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.617518 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.617518 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.617521 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.617607 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.617609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.618497 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.618498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.619874 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.619877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.620610 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.620611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.620699 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.620720 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.620801 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.620853 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":99,"thread_start_us":66,"threads_started":1}
I20260927 11:08:05.620973 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.617132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33521 (local address 127.17.95.195:56565)
0927 11:08:05.617269 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.617280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.617348 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.617517 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.617518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.617520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:05.617620 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.617621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.618497 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.618498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.619874 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.619877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.620609 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.620611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.620699 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.620720 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.620800 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.620853 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":112,"thread_start_us":49,"threads_started":1}
I20260927 11:08:05.621353 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919cc309c1444f7a286b697190c68fd" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a" is_pre_election: true
I20260927 11:08:05.621472 19666 raft_consensus.cc:2454] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 0.
I20260927 11:08:05.621506 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919cc309c1444f7a286b697190c68fd" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a20c835c1f40c782a31a27e6629cc7" is_pre_election: true
I20260927 11:08:05.621654 19806 raft_consensus.cc:2454] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 0.
I20260927 11:08:05.621649 19877 leader_election.cc:302] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8273ae9fc7044cd3894e1392421753a1, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:05.621759 20022 raft_consensus.cc:2781] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:05.621806 20022 raft_consensus.cc:482] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:05.621829 20022 raft_consensus.cc:3037] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:05.622509 20022 raft_consensus.cc:504] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.622642 20022 leader_election.cc:288] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 election: Requested vote from peers b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119), 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521)
I20260927 11:08:05.622809 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919cc309c1444f7a286b697190c68fd" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a20c835c1f40c782a31a27e6629cc7"
I20260927 11:08:05.622792 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919cc309c1444f7a286b697190c68fd" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a"
I20260927 11:08:05.622881 19666 raft_consensus.cc:3037] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:05.622881 19806 raft_consensus.cc:3037] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:05.623663 19806 raft_consensus.cc:2454] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 1.
I20260927 11:08:05.623663 19666 raft_consensus.cc:2454] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 1.
I20260927 11:08:05.623903 19877 leader_election.cc:302] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8273ae9fc7044cd3894e1392421753a1, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:05.624025 20022 raft_consensus.cc:2781] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:05.624167 20022 raft_consensus.cc:686] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [term 1 LEADER]: Becoming Leader. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Running, Role: LEADER
I20260927 11:08:05.624281 20022 consensus_queue.cc:237] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.624637 20044 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: TxnStatusTablet state changed. Reason: New leader 8273ae9fc7044cd3894e1392421753a1. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.624764 20044 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:05.624981 19508 catalog_manager.cc:5495] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } health_report { overall_health: HEALTHY } } }
I20260927 11:08:05.625288 20046 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 11:08:05.625403 20046 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 11:08:05.682966 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.680415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.0.0.1:43208)
0927 11:08:05.680503 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.680516 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.680574 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.680760 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.680761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.680762 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:05.680813 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.680815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.681560 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.681561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.682106 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.682108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.682463 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.682464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.682536 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.682550 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.682802 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.682817 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.682858 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.682905 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 11:08:05.682958 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.680483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.17.95.195:42685)
0927 11:08:05.680557 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.680559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.680569 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:05.680660 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.680661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.680677 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.680756 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:05.680882 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.681479 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.682196 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.682394 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.682464 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.682632 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.682729 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.682804 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.682861 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.682883 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 11:08:05.684633 19666 raft_consensus.cc:1261] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Refusing update from remote peer 8273ae9fc7044cd3894e1392421753a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:05.684661 19806 raft_consensus.cc:1261] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Refusing update from remote peer 8273ae9fc7044cd3894e1392421753a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:05.684983 20044 consensus_queue.cc:1035] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:05.685099 20022 consensus_queue.cc:1035] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:05.685992 20023 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: TxnStatusTablet state changed. Reason: New leader 8273ae9fc7044cd3894e1392421753a1. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.686091 20023 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:05.686265 20038 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: TxnStatusTablet state changed. Reason: New leader 8273ae9fc7044cd3894e1392421753a1. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.686353 20038 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:05.686764 20022 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.686837 20022 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:05.686908 20041 mvcc.cc:205] Tried to move back new op lower bound from 7333917842161840128 to 7333917841917652992. Current Snapshot: MvccSnapshot[applied={T|T < 7333917842161840128}]
I20260927 11:08:05.687408 20022 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.687489 20022 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 8273ae9fc7044cd3894e1392421753a1: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:05.687516 20023 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.687606 20023 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P b329ba9c33764ba184b6b1542560dc5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:05.687752 20039 mvcc.cc:205] Tried to move back new op lower bound from 7333917842161840128 to 7333917841917652992. Current Snapshot: MvccSnapshot[applied={T|T < 7333917842161840128}]
I20260927 11:08:05.688333 20037 mvcc.cc:205] Tried to move back new op lower bound from 7333917842161840128 to 7333917841917652992. Current Snapshot: MvccSnapshot[applied={T|T < 7333917842161840128}]
I20260927 11:08:05.688421 20038 tablet_replica.cc:439] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } } }
I20260927 11:08:05.688489 20038 tablet_replica.cc:442] T 2919cc309c1444f7a286b697190c68fd P 90a20c835c1f40c782a31a27e6629cc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:05.692505 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.689893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.17.95.194:33521)
0927 11:08:05.689933 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.689935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.690003 (+ 68us) server_negotiation.cc:407] Connection header received
0927 11:08:05.690076 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.690077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.690094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.690183 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:05.690260 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.690935 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.691763 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.692030 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.692108 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.692219 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.692327 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.692412 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.692429 (+ 17us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.692449 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:05.697041 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.694845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:49211 (local address 127.17.95.195:42685)
0927 11:08:05.694885 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.694888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.694994 (+ 106us) server_negotiation.cc:407] Connection header received
0927 11:08:05.695060 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.695061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.695093 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.695148 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:05.695277 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.695716 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.696383 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.696567 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.696630 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.696790 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.696819 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.696881 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.696955 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.696972 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:08:05.697068 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.694771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.17.95.194:49211)
0927 11:08:05.694915 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.694931 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.695001 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.695152 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.695152 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.695154 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:05.695214 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.695215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.695789 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.695790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.696306 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.696308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.696640 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.696641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.696698 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.696709 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.696883 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.696885 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.696950 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.696989 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260927 11:08:05.709851 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.703405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:55759 (local address 127.17.95.194:33521)
0927 11:08:05.703440 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:05.703442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:05.707291 (+ 3849us) server_negotiation.cc:407] Connection header received
0927 11:08:05.707349 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:05.707349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:05.707359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:05.707426 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:05.707629 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.708208 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.709303 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.709461 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.709503 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.709678 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.709700 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.709730 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:05.709792 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 11:08:05.709806 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:05.709940 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:05.703357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33521 (local address 127.17.95.195:55759)
0927 11:08:05.707204 (+ 3847us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:05.707217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:05.707292 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:05.707443 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:05.707444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:05.707445 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:05.707505 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.707507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.708294 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.708295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.709218 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:05.709220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:05.709524 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:05.709525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:05.709590 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:05.709603 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:05.709743 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:05.709744 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 11:08:05.709791 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 11:08:05.709829 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3798,"thread_start_us":79,"threads_started":1}
I20260927 11:08:05.719889 19508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:05.720000 19508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:05.720806 19508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Sending DeleteTablet for 3 replicas of tablet de6671e293d84dc2b95afa5fc2e7741d
I20260927 11:08:05.721125 19926 tablet_service.cc:1514] Processing DeleteTablet for tablet de6671e293d84dc2b95afa5fc2e7741d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:05 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43192
I20260927 11:08:05.721308 19786 tablet_service.cc:1514] Processing DeleteTablet for tablet de6671e293d84dc2b95afa5fc2e7741d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:05 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:32960
I20260927 11:08:05.721489 20065 tablet_replica.cc:330] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: stopping tablet replica
I20260927 11:08:05.721521 19646 tablet_service.cc:1514] Processing DeleteTablet for tablet de6671e293d84dc2b95afa5fc2e7741d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:05 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33494
I20260927 11:08:05.721671 20065 raft_consensus.cc:2229] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:05.721801 20065 raft_consensus.cc:2258] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:05.721925 19508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:05.722113 19508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:05.722543 20065 ts_tablet_manager.cc:1905] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:05.723312 19926 tablet_service.cc:1467] Processing CreateTablet for tablet 7849b2640d4d4c4fb7c37411f71c2e98 (DEFAULT_TABLE table=test-table [id=1956ab02a9394766ab7ade39a3324d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:05.723407 19786 tablet_service.cc:1467] Processing CreateTablet for tablet 7849b2640d4d4c4fb7c37411f71c2e98 (DEFAULT_TABLE table=test-table [id=1956ab02a9394766ab7ade39a3324d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:05.723446 19926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7849b2640d4d4c4fb7c37411f71c2e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.723520 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7849b2640d4d4c4fb7c37411f71c2e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.724810 20015 tablet_bootstrap.cc:492] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Bootstrap starting.
I20260927 11:08:05.724818 20016 tablet_bootstrap.cc:492] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Bootstrap starting.
I20260927 11:08:05.725212 20015 tablet_bootstrap.cc:654] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.725243 20016 tablet_bootstrap.cc:654] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.725841 19646 tablet_service.cc:1467] Processing CreateTablet for tablet 7849b2640d4d4c4fb7c37411f71c2e98 (DEFAULT_TABLE table=test-table [id=1956ab02a9394766ab7ade39a3324d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:05.725958 19646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7849b2640d4d4c4fb7c37411f71c2e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:05.727074 20065 ts_tablet_manager.cc:1918] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:05.727249 20065 log.cc:1196] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/de6671e293d84dc2b95afa5fc2e7741d
I20260927 11:08:05.727303 20017 tablet_bootstrap.cc:492] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Bootstrap starting.
I20260927 11:08:05.727656 20065 ts_tablet_manager.cc:1939] T de6671e293d84dc2b95afa5fc2e7741d P 90a20c835c1f40c782a31a27e6629cc7: Deleting consensus metadata
I20260927 11:08:05.727699 20017 tablet_bootstrap.cc:654] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:05.728164 19496 catalog_manager.cc:4841] TS 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521): tablet de6671e293d84dc2b95afa5fc2e7741d (table test-table [id=e22530aeefce4b198aa3c06780ae7142]) successfully deleted
I20260927 11:08:05.728689 20016 tablet_bootstrap.cc:492] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: No bootstrap required, opened a new log
I20260927 11:08:05.728708 20015 tablet_bootstrap.cc:492] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: No bootstrap required, opened a new log
I20260927 11:08:05.728740 20016 ts_tablet_manager.cc:1397] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 11:08:05.728762 20015 ts_tablet_manager.cc:1397] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 11:08:05.728888 20015 raft_consensus.cc:348] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.728948 20015 raft_consensus.cc:374] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.728894 20016 raft_consensus.cc:348] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.728967 20015 raft_consensus.cc:729] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.728982 20016 raft_consensus.cc:374] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.729008 20016 raft_consensus.cc:729] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.729007 20015 consensus_queue.cc:260] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.729071 20016 consensus_queue.cc:260] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.729120 20015 ts_tablet_manager.cc:1428] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:05.729164 20016 ts_tablet_manager.cc:1428] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:05.730621 20017 tablet_bootstrap.cc:492] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: No bootstrap required, opened a new log
I20260927 11:08:05.730692 20017 ts_tablet_manager.cc:1397] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 11:08:05.730839 20017 raft_consensus.cc:348] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.730906 20017 raft_consensus.cc:374] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:05.730924 20017 raft_consensus.cc:729] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b329ba9c33764ba184b6b1542560dc5a, State: Initialized, Role: FOLLOWER
I20260927 11:08:05.730959 20017 consensus_queue.cc:260] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:05.731058 20017 ts_tablet_manager.cc:1428] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:05.731283 20066 tablet_replica.cc:330] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: stopping tablet replica
I20260927 11:08:05.731442 20066 raft_consensus.cc:2229] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:05.731567 20066 raft_consensus.cc:2258] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:05.731745 20064 tablet_replica.cc:330] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: stopping tablet replica
I20260927 11:08:05.731838 20064 raft_consensus.cc:2229] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:05.731891 20064 raft_consensus.cc:2258] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:05.732275 20064 ts_tablet_manager.cc:1905] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:05.732422 20066 ts_tablet_manager.cc:1905] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:05.733901 20064 ts_tablet_manager.cc:1918] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:05.734022 20064 log.cc:1196] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/de6671e293d84dc2b95afa5fc2e7741d
I20260927 11:08:05.734303 20064 ts_tablet_manager.cc:1939] T de6671e293d84dc2b95afa5fc2e7741d P 8273ae9fc7044cd3894e1392421753a1: Deleting consensus metadata
I20260927 11:08:05.734113 20066 ts_tablet_manager.cc:1918] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:05.734573 20066 log.cc:1196] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/de6671e293d84dc2b95afa5fc2e7741d
I20260927 11:08:05.734783 19495 catalog_manager.cc:4841] TS 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685): tablet de6671e293d84dc2b95afa5fc2e7741d (table test-table [id=e22530aeefce4b198aa3c06780ae7142]) successfully deleted
I20260927 11:08:05.734936 20066 ts_tablet_manager.cc:1939] T de6671e293d84dc2b95afa5fc2e7741d P b329ba9c33764ba184b6b1542560dc5a: Deleting consensus metadata
I20260927 11:08:05.735406 19493 catalog_manager.cc:4841] TS b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119): tablet de6671e293d84dc2b95afa5fc2e7741d (table test-table [id=e22530aeefce4b198aa3c06780ae7142]) successfully deleted
I20260927 11:08:06.008213 20023 raft_consensus.cc:482] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:06.008343 20023 raft_consensus.cc:504] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:06.008850 20023 leader_election.cc:288] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521), 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685)
I20260927 11:08:06.012065 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.009062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33521 (local address 127.17.95.193:53619)
0927 11:08:06.009198 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:06.009210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:06.009281 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:06.009499 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:06.009500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:06.009502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:06.009567 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.009569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.010176 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.010177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.011195 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.011197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.011780 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.011781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.011840 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.011845 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:06.011918 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 11:08:06.011969 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260927 11:08:06.012050 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.009156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:53619 (local address 127.17.95.194:33521)
0927 11:08:06.009217 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:06.009219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:06.009262 (+ 43us) server_negotiation.cc:407] Connection header received
0927 11:08:06.009366 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:06.009367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:06.009377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:06.009459 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:06.009634 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.010092 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.011278 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.011682 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.011767 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.011777 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:06.011942 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 11:08:06.011985 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:08:06.012267 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.009074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.17.95.193:45495)
0927 11:08:06.009289 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:06.009300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:06.009384 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:06.009564 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:06.009565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:06.009567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:06.009637 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.009639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.010295 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.010296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.011314 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.011316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.011989 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.011991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.012056 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.012060 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:06.012140 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:08:06.012181 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":62,"threads_started":1}
I20260927 11:08:06.012265 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.009118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45495 (local address 127.17.95.195:42685)
0927 11:08:06.009168 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:06.009170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:06.009373 (+ 203us) server_negotiation.cc:407] Connection header received
0927 11:08:06.009459 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:06.009460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:06.009483 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:06.009561 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:06.009721 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.010185 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.011443 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.011849 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.011947 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.011966 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:06.012146 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 11:08:06.012199 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:08:06.012364 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7849b2640d4d4c4fb7c37411f71c2e98" candidate_uuid: "b329ba9c33764ba184b6b1542560dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a20c835c1f40c782a31a27e6629cc7" is_pre_election: true
I20260927 11:08:06.012460 19806 raft_consensus.cc:2454] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b329ba9c33764ba184b6b1542560dc5a in term 0.
I20260927 11:08:06.012548 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7849b2640d4d4c4fb7c37411f71c2e98" candidate_uuid: "b329ba9c33764ba184b6b1542560dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8273ae9fc7044cd3894e1392421753a1" is_pre_election: true
I20260927 11:08:06.012651 19941 raft_consensus.cc:2454] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b329ba9c33764ba184b6b1542560dc5a in term 0.
I20260927 11:08:06.012653 19600 leader_election.cc:302] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a20c835c1f40c782a31a27e6629cc7, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:06.012776 20023 raft_consensus.cc:2781] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:06.012831 20023 raft_consensus.cc:482] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:06.012877 20023 raft_consensus.cc:3037] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:06.013738 20023 raft_consensus.cc:504] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:06.013892 20023 leader_election.cc:288] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [CANDIDATE]: Term 1 election: Requested vote from peers 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521), 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685)
I20260927 11:08:06.014041 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7849b2640d4d4c4fb7c37411f71c2e98" candidate_uuid: "b329ba9c33764ba184b6b1542560dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8273ae9fc7044cd3894e1392421753a1"
I20260927 11:08:06.014078 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7849b2640d4d4c4fb7c37411f71c2e98" candidate_uuid: "b329ba9c33764ba184b6b1542560dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a20c835c1f40c782a31a27e6629cc7"
I20260927 11:08:06.014137 19806 raft_consensus.cc:3037] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:06.014139 19941 raft_consensus.cc:3037] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:06.014791 19806 raft_consensus.cc:2454] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b329ba9c33764ba184b6b1542560dc5a in term 1.
I20260927 11:08:06.014801 19941 raft_consensus.cc:2454] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b329ba9c33764ba184b6b1542560dc5a in term 1.
I20260927 11:08:06.014977 19600 leader_election.cc:302] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a20c835c1f40c782a31a27e6629cc7, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:06.015092 20023 raft_consensus.cc:2781] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:06.015226 20023 raft_consensus.cc:686] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 1 LEADER]: Becoming Leader. State: Replica: b329ba9c33764ba184b6b1542560dc5a, State: Running, Role: LEADER
I20260927 11:08:06.015300 20023 consensus_queue.cc:237] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:06.016012 19508 catalog_manager.cc:5495] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a reported cstate change: term changed from 0 to 1, leader changed from <none> to b329ba9c33764ba184b6b1542560dc5a (127.17.95.193). New cstate: current_term: 1 leader_uuid: "b329ba9c33764ba184b6b1542560dc5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:06.086525 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.084490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.17.95.193:44119)
0927 11:08:06.084549 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:06.084551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:06.084561 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:06.084616 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:06.084616 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:06.084636 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:06.084696 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:06.084771 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.085226 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.085913 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.086087 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.086137 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.086261 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:06.086357 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:06.086407 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:06.086444 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 11:08:06.086462 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:08:06.091420 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.088918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:43361 (local address 127.17.95.195:42685)
0927 11:08:06.088964 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:06.088965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:06.089115 (+ 150us) server_negotiation.cc:407] Connection header received
0927 11:08:06.089150 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:06.089151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:06.089158 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:06.089197 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:06.089638 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.090071 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.090835 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.091018 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.091111 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.091227 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:06.091259 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:06.091311 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:06.091354 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 11:08:06.091369 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 11:08:06.091446 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.088845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.17.95.193:43361)
0927 11:08:06.089044 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:06.089052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:06.089102 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:06.089210 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:06.089211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:06.089212 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:06.089576 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.089578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.090157 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.090158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.090769 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.090771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.091097 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.091098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.091140 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.091151 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:06.091318 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:06.091319 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 11:08:06.091352 (+ 33us) client_negotiation.cc:240] Negotiation successful
0927 11:08:06.091383 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260927 11:08:06.093297 19941 raft_consensus.cc:1261] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Refusing update from remote peer b329ba9c33764ba184b6b1542560dc5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:06.093501 19806 raft_consensus.cc:1261] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Refusing update from remote peer b329ba9c33764ba184b6b1542560dc5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:06.093596 20072 consensus_queue.cc:1035] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:06.093734 20023 consensus_queue.cc:1035] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:06.100059 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.097922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:52217 (local address 127.17.95.193:44119)
0927 11:08:06.097960 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:06.097962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:06.097969 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:06.098042 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:06.098043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:06.098053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:06.098104 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:06.098227 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.098727 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.099451 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.099628 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.099711 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.099836 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:06.099868 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:06.099914 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:06.099956 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 11:08:06.099976 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:06.100082 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:06.097863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44119 (local address 127.17.95.195:52217)
0927 11:08:06.097931 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:06.097942 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:06.097978 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:06.098122 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:06.098123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:06.098124 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:06.098165 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.098167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.098790 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.098790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.099384 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:06.099386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:06.099712 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:06.099713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:06.099769 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:06.099775 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:06.099912 (+ 137us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:06.099914 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:06.099955 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 11:08:06.100003 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
W20260927 11:08:06.100378 20039 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 11:08:07.619122 19507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:07.619215 19507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:07.620054 19507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Sending DeleteTablet for 3 replicas of tablet 7849b2640d4d4c4fb7c37411f71c2e98
I20260927 11:08:07.620306 19926 tablet_service.cc:1514] Processing DeleteTablet for tablet 7849b2640d4d4c4fb7c37411f71c2e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43192
I20260927 11:08:07.620304 19786 tablet_service.cc:1514] Processing DeleteTablet for tablet 7849b2640d4d4c4fb7c37411f71c2e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:32960
I20260927 11:08:07.620318 19646 tablet_service.cc:1514] Processing DeleteTablet for tablet 7849b2640d4d4c4fb7c37411f71c2e98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33494
I20260927 11:08:07.620479 20102 tablet_replica.cc:330] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: stopping tablet replica
I20260927 11:08:07.620536 20102 raft_consensus.cc:2229] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:07.620579 20102 raft_consensus.cc:2258] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:07.620579 20104 tablet_replica.cc:330] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: stopping tablet replica
I20260927 11:08:07.620641 20104 raft_consensus.cc:2229] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:07.620730 20104 raft_consensus.cc:2258] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:07.620831 20102 ts_tablet_manager.cc:1905] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:07.620920 20104 ts_tablet_manager.cc:1905] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:07.622290 20104 ts_tablet_manager.cc:1918] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 11:08:07.622362 20104 log.cc:1196] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/7849b2640d4d4c4fb7c37411f71c2e98
I20260927 11:08:07.622668 20104 ts_tablet_manager.cc:1939] T 7849b2640d4d4c4fb7c37411f71c2e98 P b329ba9c33764ba184b6b1542560dc5a: Deleting consensus metadata
I20260927 11:08:07.622977 19493 catalog_manager.cc:4841] TS b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119): tablet 7849b2640d4d4c4fb7c37411f71c2e98 (table test-table [id=1956ab02a9394766ab7ade39a3324d85]) successfully deleted
I20260927 11:08:07.623111 20102 ts_tablet_manager.cc:1918] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 11:08:07.623179 20102 log.cc:1196] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/7849b2640d4d4c4fb7c37411f71c2e98
I20260927 11:08:07.623406 20102 ts_tablet_manager.cc:1939] T 7849b2640d4d4c4fb7c37411f71c2e98 P 8273ae9fc7044cd3894e1392421753a1: Deleting consensus metadata
I20260927 11:08:07.623718 19495 catalog_manager.cc:4841] TS 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685): tablet 7849b2640d4d4c4fb7c37411f71c2e98 (table test-table [id=1956ab02a9394766ab7ade39a3324d85]) successfully deleted
I20260927 11:08:07.624632 20103 tablet_replica.cc:330] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: stopping tablet replica
I20260927 11:08:07.624701 20103 raft_consensus.cc:2229] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:07.624744 20103 raft_consensus.cc:2258] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:07.624970 20103 ts_tablet_manager.cc:1905] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:07.626845 20103 ts_tablet_manager.cc:1918] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 11:08:07.626909 20103 log.cc:1196] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/7849b2640d4d4c4fb7c37411f71c2e98
I20260927 11:08:07.627153 20103 ts_tablet_manager.cc:1939] T 7849b2640d4d4c4fb7c37411f71c2e98 P 90a20c835c1f40c782a31a27e6629cc7: Deleting consensus metadata
I20260927 11:08:07.627509 19496 catalog_manager.cc:4841] TS 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521): tablet 7849b2640d4d4c4fb7c37411f71c2e98 (table test-table [id=1956ab02a9394766ab7ade39a3324d85]) successfully deleted
Sep 27 11:08:07 dist-test-slave-6j0v krb5kdc[19464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 11:08:07 dist-test-slave-6j0v krb5kdc[19464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:07.651291 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:07.638927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.17.95.254:34361)
0927 11:08:07.639114 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:07.639117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:07.639127 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:07.639206 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:07.639207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:07.639218 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:07.639284 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:07.639894 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.640393 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.641147 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.641331 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.641378 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:07.648607 (+ 7229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:07.648608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:07.648609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:07.648615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:07.650169 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:07.650556 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:07.650557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:07.650557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:07.650611 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:07.650813 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:07.650814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:07.650814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:07.650920 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:07.650996 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:07.651149 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 11:08:07.651217 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":91,"threads_started":1}
I20260927 11:08:07.652993 19507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:07.653184 19507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:07.654609 19926 tablet_service.cc:1467] Processing CreateTablet for tablet 14e975b0c2b04563871d326db68d1393 (DEFAULT_TABLE table=test-table [id=ab4a0612b3f44a53ae4e7dc56ad7b55e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:07.654620 19786 tablet_service.cc:1467] Processing CreateTablet for tablet 14e975b0c2b04563871d326db68d1393 (DEFAULT_TABLE table=test-table [id=ab4a0612b3f44a53ae4e7dc56ad7b55e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:07.654778 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e975b0c2b04563871d326db68d1393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:07.654775 19926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e975b0c2b04563871d326db68d1393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:07.656011 20113 tablet_bootstrap.cc:492] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Bootstrap starting.
I20260927 11:08:07.656066 19646 tablet_service.cc:1467] Processing CreateTablet for tablet 14e975b0c2b04563871d326db68d1393 (DEFAULT_TABLE table=test-table [id=ab4a0612b3f44a53ae4e7dc56ad7b55e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:07.656190 19646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e975b0c2b04563871d326db68d1393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:07.656386 20113 tablet_bootstrap.cc:654] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:07.656684 20114 tablet_bootstrap.cc:492] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Bootstrap starting.
I20260927 11:08:07.656966 20113 tablet_bootstrap.cc:492] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: No bootstrap required, opened a new log
I20260927 11:08:07.657004 20114 tablet_bootstrap.cc:654] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:07.657035 20113 ts_tablet_manager.cc:1397] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:07.657199 20113 raft_consensus.cc:348] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.657279 20113 raft_consensus.cc:374] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:07.657313 20113 raft_consensus.cc:729] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Initialized, Role: FOLLOWER
I20260927 11:08:07.657354 20113 consensus_queue.cc:260] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.657541 20113 ts_tablet_manager.cc:1428] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:07.657691 20114 tablet_bootstrap.cc:492] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: No bootstrap required, opened a new log
I20260927 11:08:07.657748 20114 ts_tablet_manager.cc:1397] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:07.657877 20114 raft_consensus.cc:348] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.657956 20114 raft_consensus.cc:374] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:07.657984 20114 raft_consensus.cc:729] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Initialized, Role: FOLLOWER
I20260927 11:08:07.658028 20114 consensus_queue.cc:260] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.658162 20114 ts_tablet_manager.cc:1428] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:07.658265 20118 tablet_bootstrap.cc:492] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Bootstrap starting.
I20260927 11:08:07.658603 20118 tablet_bootstrap.cc:654] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:07.659246 20118 tablet_bootstrap.cc:492] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: No bootstrap required, opened a new log
I20260927 11:08:07.659289 20118 ts_tablet_manager.cc:1397] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:07.659425 20118 raft_consensus.cc:348] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.659559 20118 raft_consensus.cc:374] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:07.659583 20118 raft_consensus.cc:729] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b329ba9c33764ba184b6b1542560dc5a, State: Initialized, Role: FOLLOWER
I20260927 11:08:07.659633 20118 consensus_queue.cc:260] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.659731 20118 ts_tablet_manager.cc:1428] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:07.691721 20117 raft_consensus.cc:482] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:07.691808 20117 raft_consensus.cc:504] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.692042 20117 leader_election.cc:288] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119), 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685)
I20260927 11:08:07.692199 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e975b0c2b04563871d326db68d1393" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a" is_pre_election: true
I20260927 11:08:07.692214 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e975b0c2b04563871d326db68d1393" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8273ae9fc7044cd3894e1392421753a1" is_pre_election: true
I20260927 11:08:07.692325 19941 raft_consensus.cc:2454] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 0.
I20260927 11:08:07.692324 19666 raft_consensus.cc:2454] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 0.
I20260927 11:08:07.692472 19737 leader_election.cc:302] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a20c835c1f40c782a31a27e6629cc7, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:07.692642 20117 raft_consensus.cc:2781] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:07.692695 20117 raft_consensus.cc:482] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:07.692710 20117 raft_consensus.cc:3037] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:07.693403 20117 raft_consensus.cc:504] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.693536 20117 leader_election.cc:288] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 election: Requested vote from peers b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119), 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685)
I20260927 11:08:07.693683 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e975b0c2b04563871d326db68d1393" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a"
I20260927 11:08:07.693698 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e975b0c2b04563871d326db68d1393" candidate_uuid: "90a20c835c1f40c782a31a27e6629cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8273ae9fc7044cd3894e1392421753a1"
I20260927 11:08:07.693764 19941 raft_consensus.cc:3037] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:07.693764 19666 raft_consensus.cc:3037] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:07.694520 19666 raft_consensus.cc:2454] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 1.
I20260927 11:08:07.694520 19941 raft_consensus.cc:2454] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a20c835c1f40c782a31a27e6629cc7 in term 1.
I20260927 11:08:07.694708 19739 leader_election.cc:302] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8273ae9fc7044cd3894e1392421753a1, 90a20c835c1f40c782a31a27e6629cc7; no voters:
I20260927 11:08:07.694819 20117 raft_consensus.cc:2781] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:07.695005 20117 raft_consensus.cc:686] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 LEADER]: Becoming Leader. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Running, Role: LEADER
I20260927 11:08:07.695107 20117 consensus_queue.cc:237] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.695657 19507 catalog_manager.cc:5495] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194). New cstate: current_term: 1 leader_uuid: "90a20c835c1f40c782a31a27e6629cc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:07.708786 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:07.705688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.17.95.194:33521)
0927 11:08:07.705826 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:07.705828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:07.705835 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:07.705855 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:07.705856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:07.705886 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:07.705987 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:07.706152 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.706640 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.707993 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.708213 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.708300 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:07.708503 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:07.708539 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:07.708613 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:07.708677 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 11:08:07.708699 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260927 11:08:07.710837 19666 raft_consensus.cc:1261] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Refusing update from remote peer 90a20c835c1f40c782a31a27e6629cc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:07.710845 19941 raft_consensus.cc:1261] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Refusing update from remote peer 90a20c835c1f40c782a31a27e6629cc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:07.711054 20120 consensus_queue.cc:1035] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:07.711176 20117 consensus_queue.cc:1035] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:07.720870 19507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:07.720947 19507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:07.721539 19507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Sending DeleteTablet for 3 replicas of tablet 14e975b0c2b04563871d326db68d1393
I20260927 11:08:07.721807 19926 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e975b0c2b04563871d326db68d1393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43192
I20260927 11:08:07.721807 19786 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e975b0c2b04563871d326db68d1393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:32960
I20260927 11:08:07.721894 20103 tablet_replica.cc:330] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: stopping tablet replica
I20260927 11:08:07.721892 20102 tablet_replica.cc:330] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: stopping tablet replica
I20260927 11:08:07.721940 20103 raft_consensus.cc:2229] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:07.721942 20102 raft_consensus.cc:2229] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:07.721992 20103 raft_consensus.cc:2258] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:07.721997 20102 raft_consensus.cc:2258] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:07.722011 19507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 11:08:07.722110 19646 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e975b0c2b04563871d326db68d1393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33494
W20260927 11:08:07.722148 19507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:07.722216 20104 tablet_replica.cc:330] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: stopping tablet replica
I20260927 11:08:07.722281 20104 raft_consensus.cc:2229] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:07.722332 20104 raft_consensus.cc:2258] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:07.722419 20103 ts_tablet_manager.cc:1905] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:07.722471 20102 ts_tablet_manager.cc:1905] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:07.723536 19646 tablet_service.cc:1467] Processing CreateTablet for tablet 364d0e40091840219db77fd412d4018b (DEFAULT_TABLE table=test-table [id=16ef23f4ed13435f9475868c67744240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:07.723631 19646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d0e40091840219db77fd412d4018b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:07.724025 20102 ts_tablet_manager.cc:1918] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:07.724092 20102 log.cc:1196] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/14e975b0c2b04563871d326db68d1393
I20260927 11:08:07.724086 20103 ts_tablet_manager.cc:1918] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:07.724126 20103 log.cc:1196] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/14e975b0c2b04563871d326db68d1393
I20260927 11:08:07.724283 20103 ts_tablet_manager.cc:1939] T 14e975b0c2b04563871d326db68d1393 P 90a20c835c1f40c782a31a27e6629cc7: Deleting consensus metadata
I20260927 11:08:07.724311 20102 ts_tablet_manager.cc:1939] T 14e975b0c2b04563871d326db68d1393 P 8273ae9fc7044cd3894e1392421753a1: Deleting consensus metadata
I20260927 11:08:07.724604 19786 tablet_service.cc:1467] Processing CreateTablet for tablet 364d0e40091840219db77fd412d4018b (DEFAULT_TABLE table=test-table [id=16ef23f4ed13435f9475868c67744240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:07.724725 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d0e40091840219db77fd412d4018b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:07.724704 19926 tablet_service.cc:1467] Processing CreateTablet for tablet 364d0e40091840219db77fd412d4018b (DEFAULT_TABLE table=test-table [id=16ef23f4ed13435f9475868c67744240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:07.724797 19926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d0e40091840219db77fd412d4018b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:07.725026 20118 tablet_bootstrap.cc:492] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a: Bootstrap starting.
I20260927 11:08:07.725301 20118 tablet_bootstrap.cc:654] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:07.725788 20118 tablet_bootstrap.cc:492] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a: No bootstrap required, opened a new log
I20260927 11:08:07.725855 20118 ts_tablet_manager.cc:1397] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 11:08:07.725874 20113 tablet_bootstrap.cc:492] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Bootstrap starting.
I20260927 11:08:07.725947 20118 raft_consensus.cc:348] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.725973 20104 ts_tablet_manager.cc:1905] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:07.726008 20118 raft_consensus.cc:374] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:07.726028 20118 raft_consensus.cc:729] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b329ba9c33764ba184b6b1542560dc5a, State: Initialized, Role: FOLLOWER
I20260927 11:08:07.726068 20118 consensus_queue.cc:260] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.726166 20113 tablet_bootstrap.cc:654] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:07.726169 20118 ts_tablet_manager.cc:1428] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:07.727156 20114 tablet_bootstrap.cc:492] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Bootstrap starting.
I20260927 11:08:07.727494 20104 ts_tablet_manager.cc:1918] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:07.727514 20114 tablet_bootstrap.cc:654] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:07.727564 20104 log.cc:1196] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/14e975b0c2b04563871d326db68d1393
I20260927 11:08:07.727821 20104 ts_tablet_manager.cc:1939] T 14e975b0c2b04563871d326db68d1393 P b329ba9c33764ba184b6b1542560dc5a: Deleting consensus metadata
I20260927 11:08:07.727960 19496 catalog_manager.cc:4841] TS 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521): tablet 14e975b0c2b04563871d326db68d1393 (table test-table [id=ab4a0612b3f44a53ae4e7dc56ad7b55e]) successfully deleted
I20260927 11:08:07.728084 19495 catalog_manager.cc:4841] TS 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685): tablet 14e975b0c2b04563871d326db68d1393 (table test-table [id=ab4a0612b3f44a53ae4e7dc56ad7b55e]) successfully deleted
I20260927 11:08:07.728209 19493 catalog_manager.cc:4841] TS b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119): tablet 14e975b0c2b04563871d326db68d1393 (table test-table [id=ab4a0612b3f44a53ae4e7dc56ad7b55e]) successfully deleted
I20260927 11:08:07.728437 20113 tablet_bootstrap.cc:492] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: No bootstrap required, opened a new log
I20260927 11:08:07.728440 20114 tablet_bootstrap.cc:492] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: No bootstrap required, opened a new log
I20260927 11:08:07.728487 20113 ts_tablet_manager.cc:1397] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260927 11:08:07.728511 20114 ts_tablet_manager.cc:1397] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:07.728590 20113 raft_consensus.cc:348] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.728657 20113 raft_consensus.cc:374] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:07.728672 20113 raft_consensus.cc:729] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a20c835c1f40c782a31a27e6629cc7, State: Initialized, Role: FOLLOWER
I20260927 11:08:07.728646 20114 raft_consensus.cc:348] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.728734 20114 raft_consensus.cc:374] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:07.728725 20113 consensus_queue.cc:260] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.728761 20114 raft_consensus.cc:729] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Initialized, Role: FOLLOWER
I20260927 11:08:07.728806 20113 ts_tablet_manager.cc:1428] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:07.728809 20114 consensus_queue.cc:260] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.728891 20114 ts_tablet_manager.cc:1428] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:07.865765 20083 raft_consensus.cc:482] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:07.865872 20083 raft_consensus.cc:504] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.866044 20083 leader_election.cc:288] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119), 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521)
I20260927 11:08:07.866284 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d0e40091840219db77fd412d4018b" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a20c835c1f40c782a31a27e6629cc7" is_pre_election: true
I20260927 11:08:07.866309 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d0e40091840219db77fd412d4018b" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a" is_pre_election: true
I20260927 11:08:07.866384 19806 raft_consensus.cc:2454] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 0.
I20260927 11:08:07.866432 19666 raft_consensus.cc:2454] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 0.
I20260927 11:08:07.866557 19880 leader_election.cc:302] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8273ae9fc7044cd3894e1392421753a1, 90a20c835c1f40c782a31a27e6629cc7; no voters:
I20260927 11:08:07.866676 20083 raft_consensus.cc:2781] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:07.866775 20083 raft_consensus.cc:482] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:07.866808 20083 raft_consensus.cc:3037] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:07.867754 20083 raft_consensus.cc:504] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.867930 20083 leader_election.cc:288] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 election: Requested vote from peers b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119), 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521)
I20260927 11:08:07.868115 19666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d0e40091840219db77fd412d4018b" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b329ba9c33764ba184b6b1542560dc5a"
I20260927 11:08:07.868121 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d0e40091840219db77fd412d4018b" candidate_uuid: "8273ae9fc7044cd3894e1392421753a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a20c835c1f40c782a31a27e6629cc7"
I20260927 11:08:07.868206 19666 raft_consensus.cc:3037] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:07.868242 19806 raft_consensus.cc:3037] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:07.868849 19666 raft_consensus.cc:2454] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 1.
I20260927 11:08:07.868983 19806 raft_consensus.cc:2454] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8273ae9fc7044cd3894e1392421753a1 in term 1.
I20260927 11:08:07.869104 19877 leader_election.cc:302] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8273ae9fc7044cd3894e1392421753a1, b329ba9c33764ba184b6b1542560dc5a; no voters:
I20260927 11:08:07.869199 20083 raft_consensus.cc:2781] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:07.869249 20083 raft_consensus.cc:686] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 1 LEADER]: Becoming Leader. State: Replica: 8273ae9fc7044cd3894e1392421753a1, State: Running, Role: LEADER
I20260927 11:08:07.869302 20083 consensus_queue.cc:237] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } }
I20260927 11:08:07.869897 19507 catalog_manager.cc:5495] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "8273ae9fc7044cd3894e1392421753a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8273ae9fc7044cd3894e1392421753a1" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 42685 } health_report { overall_health: HEALTHY } } }
I20260927 11:08:07.913439 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:07.909097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.17.95.195:42685)
0927 11:08:07.909290 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:07.909292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:07.910695 (+ 1403us) server_negotiation.cc:407] Connection header received
0927 11:08:07.910799 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:07.910800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:07.910815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:07.910925 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:07.911095 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.911655 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.912517 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.912820 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.912918 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:07.913104 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:07.913167 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:07.913243 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:07.913316 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 11:08:07.913341 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260927 11:08:07.918120 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:07.915956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:48213 (local address 127.17.95.195:42685)
0927 11:08:07.916030 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:07.916032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:07.916042 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:07.916129 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:07.916130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:07.916137 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:07.916185 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:07.916317 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.916737 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.917589 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.917757 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.917806 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:07.917919 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:07.917941 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:07.917991 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:07.918029 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 11:08:07.918045 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 11:08:07.918128 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:07.915840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:42685 (local address 127.17.95.195:48213)
0927 11:08:07.915999 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:07.916009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:07.916055 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:07.916198 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:07.916198 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:07.916200 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:07.916257 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:07.916258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.916888 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.916889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:07.917499 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:07.917500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:07.917824 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:07.917825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:07.917863 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:07.917866 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:07.917997 (+ 131us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:07.917998 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 11:08:07.918029 (+ 31us) client_negotiation.cc:240] Negotiation successful
0927 11:08:07.918063 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260927 11:08:07.920152 19666 raft_consensus.cc:1261] T 364d0e40091840219db77fd412d4018b P b329ba9c33764ba184b6b1542560dc5a [term 1 FOLLOWER]: Refusing update from remote peer 8273ae9fc7044cd3894e1392421753a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:07.920152 19806 raft_consensus.cc:1261] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Refusing update from remote peer 8273ae9fc7044cd3894e1392421753a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:07.920416 20022 consensus_queue.cc:1035] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b329ba9c33764ba184b6b1542560dc5a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:07.920508 20088 consensus_queue.cc:1035] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a20c835c1f40c782a31a27e6629cc7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 11:08:07.925175 20081 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 11:08:09.442430 19508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:09.442536 19508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:09.443290 19508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 181de01f62da45e5bb7b59f932a356d4: Sending DeleteTablet for 3 replicas of tablet 364d0e40091840219db77fd412d4018b
I20260927 11:08:09.443547 19786 tablet_service.cc:1514] Processing DeleteTablet for tablet 364d0e40091840219db77fd412d4018b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:09 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:32960
I20260927 11:08:09.443545 19646 tablet_service.cc:1514] Processing DeleteTablet for tablet 364d0e40091840219db77fd412d4018b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:09 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33494
I20260927 11:08:09.443696 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 19581
I20260927 11:08:09.443770 19926 tablet_service.cc:1514] Processing DeleteTablet for tablet 364d0e40091840219db77fd412d4018b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:09 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43192
I20260927 11:08:09.445348 20161 tablet_replica.cc:330] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: stopping tablet replica
I20260927 11:08:09.445413 20161 raft_consensus.cc:2229] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:09.445464 20161 raft_consensus.cc:2258] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:09.445580 20162 tablet_replica.cc:330] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: stopping tablet replica
I20260927 11:08:09.445633 20162 raft_consensus.cc:2229] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:09.445720 20162 raft_consensus.cc:2258] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:09.445740 20161 ts_tablet_manager.cc:1905] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:09.445938 20162 ts_tablet_manager.cc:1905] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:09.447508 20161 ts_tablet_manager.cc:1918] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 11:08:09.447559 20161 log.cc:1196] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/364d0e40091840219db77fd412d4018b
I20260927 11:08:09.447719 20161 ts_tablet_manager.cc:1939] T 364d0e40091840219db77fd412d4018b P 90a20c835c1f40c782a31a27e6629cc7: Deleting consensus metadata
I20260927 11:08:09.447911 20162 ts_tablet_manager.cc:1918] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 11:08:09.447976 20162 log.cc:1196] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/364d0e40091840219db77fd412d4018b
I20260927 11:08:09.448015 19496 catalog_manager.cc:4841] TS 90a20c835c1f40c782a31a27e6629cc7 (127.17.95.194:33521): tablet 364d0e40091840219db77fd412d4018b (table test-table [id=16ef23f4ed13435f9475868c67744240]) successfully deleted
I20260927 11:08:09.448215 20162 ts_tablet_manager.cc:1939] T 364d0e40091840219db77fd412d4018b P 8273ae9fc7044cd3894e1392421753a1: Deleting consensus metadata
I20260927 11:08:09.448493 19495 catalog_manager.cc:4841] TS 8273ae9fc7044cd3894e1392421753a1 (127.17.95.195:42685): tablet 364d0e40091840219db77fd412d4018b (table test-table [id=16ef23f4ed13435f9475868c67744240]) successfully deleted
W20260927 11:08:09.451519 19493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:44119 (error 108)
I20260927 11:08:09.451649 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 19721
I20260927 11:08:09.452486 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:09.452268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44119 (local address 127.0.0.1:49078)
0927 11:08:09.452394 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:09.452440 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:44119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
W20260927 11:08:09.452584 19493 catalog_manager.cc:4584] TS b329ba9c33764ba184b6b1542560dc5a (127.17.95.193:44119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 364d0e40091840219db77fd412d4018b: Network error: Client connection negotiation failed: client connection to 127.17.95.193:44119: connect: Connection refused (error 111)
I20260927 11:08:09.457408 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 19861
I20260927 11:08:09.462783 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 19477
2026-09-27T11:08:09Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6137 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:09 dist-test-slave-6j0v krb5kdc[20167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:09 dist-test-slave-6j0v krb5kdc[20167](info): set up 2 sockets
Sep 27 11:08:09 dist-test-slave-6j0v krb5kdc[20167](info): commencing operation
krb5kdc: starting...
W20260927 11:08:10.499857 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:10 dist-test-slave-6j0v krb5kdc[20167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:10.609553 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:42459
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:10.686012 20180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:10.686221 20180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:10.686257 20180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:10.687433 20180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:10.687497 20180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:10.687521 20180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:10.687549 20180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:10.687580 20180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:10.688997 20180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:10.689262 20180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:10.689482 20180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:10.694535 20185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:10.694552 20180 server_base.cc:1034] running on GCE node
W20260927 11:08:10.694546 20188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:10.694535 20186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:10.694900 20180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:10.695235 20180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:10.696400 20180 hybrid_clock.cc:648] HybridClock initialized: now 1790507290696380 us; error 39 us; skew 500 ppm
Sep 27 11:08:10 dist-test-slave-6j0v krb5kdc[20167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507290, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:10.698985 20180 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:10.699411 20180 webserver.cc:460] Webserver started at http://127.17.95.254:36635/ using document root <none> and password file <none>
I20260927 11:08:10.699608 20180 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:10.699646 20180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:10.699733 20180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:10.700695 20180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "9064ddb834c741ea9e60f0ecde7948ea"
format_stamp: "Formatted at 2026-09-27 11:08:10 on dist-test-slave-6j0v"
server_key: "f1b88100b913207c42ff90412b8d683e"
server_key_iv: "31659d8f2b9c8b9652be74dcf639e50e"
server_key_version: "encryptionkey@0"
I20260927 11:08:10.700985 20180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "9064ddb834c741ea9e60f0ecde7948ea"
format_stamp: "Formatted at 2026-09-27 11:08:10 on dist-test-slave-6j0v"
server_key: "f1b88100b913207c42ff90412b8d683e"
server_key_iv: "31659d8f2b9c8b9652be74dcf639e50e"
server_key_version: "encryptionkey@0"
I20260927 11:08:10.702243 20180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:10.702920 20195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:10.703116 20180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:10.703197 20180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "9064ddb834c741ea9e60f0ecde7948ea"
format_stamp: "Formatted at 2026-09-27 11:08:10 on dist-test-slave-6j0v"
server_key: "f1b88100b913207c42ff90412b8d683e"
server_key_iv: "31659d8f2b9c8b9652be74dcf639e50e"
server_key_version: "encryptionkey@0"
I20260927 11:08:10.703248 20180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:10.720348 20180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:10.721972 20180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:10.722072 20180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:10.725977 20180 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:42459
I20260927 11:08:10.726045 20247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:42459 every 8 connection(s)
I20260927 11:08:10.726383 20180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:10.727275 20248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:10.729342 20248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea: Bootstrap starting.
I20260927 11:08:10.729862 20248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:10.730131 20248 log.cc:824] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:10.730808 20248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea: No bootstrap required, opened a new log
I20260927 11:08:10.732455 20248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9064ddb834c741ea9e60f0ecde7948ea" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42459 } }
I20260927 11:08:10.732625 20248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:10.732669 20248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9064ddb834c741ea9e60f0ecde7948ea, State: Initialized, Role: FOLLOWER
I20260927 11:08:10.732824 20248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9064ddb834c741ea9e60f0ecde7948ea" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42459 } }
I20260927 11:08:10.732928 20248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:10.732968 20248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:10.733023 20248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:10.733376 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20180
I20260927 11:08:10.733466 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:10.733644 17791 external_mini_cluster.cc:1428] Setting key db92ab2a93390a5668d5ba6b01a74214
I20260927 11:08:10.733712 20248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9064ddb834c741ea9e60f0ecde7948ea" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42459 } }
I20260927 11:08:10.733826 20248 leader_election.cc:302] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9064ddb834c741ea9e60f0ecde7948ea; no voters:
I20260927 11:08:10.733964 20248 leader_election.cc:288] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:10.733986 20251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:10.734098 20251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [term 1 LEADER]: Becoming Leader. State: Replica: 9064ddb834c741ea9e60f0ecde7948ea, State: Running, Role: LEADER
I20260927 11:08:10.734162 20251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9064ddb834c741ea9e60f0ecde7948ea" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42459 } }
I20260927 11:08:10.734193 20248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:10.734448 20251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9064ddb834c741ea9e60f0ecde7948ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9064ddb834c741ea9e60f0ecde7948ea" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42459 } } }
I20260927 11:08:10.734531 20251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:10.734650 20251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9064ddb834c741ea9e60f0ecde7948ea. Latest consensus state: current_term: 1 leader_uuid: "9064ddb834c741ea9e60f0ecde7948ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9064ddb834c741ea9e60f0ecde7948ea" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42459 } } }
I20260927 11:08:10.734683 20251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:10.734787 20257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:10.735316 20257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:10.737102 20257 catalog_manager.cc:1347] Generated new cluster ID: 782ae2b2eb5f43669305ca4ad5a1fc99
I20260927 11:08:10.737152 20257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:08:10 dist-test-slave-6j0v krb5kdc[20167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:10.745344 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:10.734086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46774 (local address 127.17.95.254:42459)
0927 11:08:10.734333 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:10.734342 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:10.734368 (+ 26us) server_negotiation.cc:407] Connection header received
0927 11:08:10.734519 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:10.734526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:10.734577 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:10.735545 (+ 968us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:10.735637 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:10.736977 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:10.738029 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:10.738655 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:10.738764 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:10.741402 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:10.741406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:10.741406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:10.741418 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:10.743757 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:10.744140 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:10.744142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:10.744142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:10.744253 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:10.744461 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:10.744462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:10.744462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:10.744734 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:10.744894 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:10.745029 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 11:08:10.745148 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260927 11:08:10.747363 20257 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:10.747839 20257 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:10.753114 20257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9064ddb834c741ea9e60f0ecde7948ea: Generated new TSK 0
I20260927 11:08:10.753286 20257 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:10.780910 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42459
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:10.874023 20274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:10.874250 20274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:10.874287 20274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:10.875514 20274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:10.875571 20274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:10.875617 20274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:10.877002 20274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:10.877228 20274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:10.877429 20274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:10.882251 20279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:10.882273 20280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:10.882540 20274 server_base.cc:1034] running on GCE node
W20260927 11:08:10.882300 20282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:10.882861 20274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:10.883147 20274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:10.884315 20274 hybrid_clock.cc:648] HybridClock initialized: now 1790507290884296 us; error 30 us; skew 500 ppm
Sep 27 11:08:10 dist-test-slave-6j0v krb5kdc[20167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507290, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:10.887334 20274 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:10.887764 20274 webserver.cc:460] Webserver started at http://127.17.95.193:34645/ using document root <none> and password file <none>
I20260927 11:08:10.887979 20274 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:10.888051 20274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:10.888195 20274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:10.889185 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "d83596958cc34fa983fce2bea01003c3"
format_stamp: "Formatted at 2026-09-27 11:08:10 on dist-test-slave-6j0v"
server_key: "dbf8f5a98f885e3b9a9f9f336c35654e"
server_key_iv: "7a1f06b04eb472b8dac1e58d2889d8e7"
server_key_version: "encryptionkey@0"
I20260927 11:08:10.889511 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "d83596958cc34fa983fce2bea01003c3"
format_stamp: "Formatted at 2026-09-27 11:08:10 on dist-test-slave-6j0v"
server_key: "dbf8f5a98f885e3b9a9f9f336c35654e"
server_key_iv: "7a1f06b04eb472b8dac1e58d2889d8e7"
server_key_version: "encryptionkey@0"
I20260927 11:08:10.890863 20274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:10.891682 20289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:10.891846 20274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:10.891963 20274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "d83596958cc34fa983fce2bea01003c3"
format_stamp: "Formatted at 2026-09-27 11:08:10 on dist-test-slave-6j0v"
server_key: "dbf8f5a98f885e3b9a9f9f336c35654e"
server_key_iv: "7a1f06b04eb472b8dac1e58d2889d8e7"
server_key_version: "encryptionkey@0"
I20260927 11:08:10.892057 20274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:10.907262 20274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:10.909015 20274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:10.909165 20274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:10.909374 20274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:10.909662 20274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:10.909721 20274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:10.909759 20274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:10.909786 20274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:10.915377 20274 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:44963
I20260927 11:08:10.915450 20402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:44963 every 8 connection(s)
I20260927 11:08:10.915759 20274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:10 dist-test-slave-6j0v krb5kdc[20167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507290, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:10.924249 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:10.916411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:55471 (local address 127.17.95.254:42459)
0927 11:08:10.916479 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:10.916481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:10.916709 (+ 228us) server_negotiation.cc:407] Connection header received
0927 11:08:10.917199 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:10.917201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:10.917214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:10.917264 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:10.918123 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:10.918613 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:10.919605 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:10.919814 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:10.919894 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:10.921870 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:10.921871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:10.921871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:10.921877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:10.923040 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:10.923452 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:10.923452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:10.923453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:10.923508 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:10.923705 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:10.923706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:10.923706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:10.923799 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:10.923846 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:10.924072 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 11:08:10.924164 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 11:08:10.924404 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:10.916414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42459 (local address 127.17.95.193:55471)
0927 11:08:10.916599 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:10.916632 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:10.917103 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:10.917332 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:10.917336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:10.917851 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:10.918027 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:10.918032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:10.918698 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:10.918699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:10.919507 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:10.919509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:10.919902 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:10.919903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:10.920038 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:10.920528 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:10.920535 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:10.921784 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:10.923186 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:10.923192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:10.923193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:10.923390 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:10.923564 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:10.923564 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:10.923565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:10.923654 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:10.923849 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:10.923853 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:10.923954 (+ 101us) client_negotiation.cc:769] Sending connection context
0927 11:08:10.924034 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:08:10.924150 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260927 11:08:10.925014 20403 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42459
I20260927 11:08:10.925122 20403 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:10.925266 20403 heartbeater.cc:511] Master 127.17.95.254:42459 requested a full tablet report, sending...
I20260927 11:08:10.925362 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20274
I20260927 11:08:10.925467 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:10.925658 17791 external_mini_cluster.cc:1428] Setting key f1d2df83a5a27411b0b5b519461f4f64
I20260927 11:08:10.925845 20212 ts_manager.cc:194] Registered new tserver with Master: d83596958cc34fa983fce2bea01003c3 (127.17.95.193:44963)
I20260927 11:08:10.926698 20212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:55471
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:10.950481 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42459
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:11.029538 20408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:11.029728 20408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:11.029752 20408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:11.030906 20408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:11.030946 20408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:11.030972 20408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:11.032387 20408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:11.032621 20408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:11.032837 20408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:11.037962 20416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:11.037978 20413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:11.037981 20414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:11.038002 20408 server_base.cc:1034] running on GCE node
I20260927 11:08:11.038362 20408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:11.038549 20408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:11.039726 20408 hybrid_clock.cc:648] HybridClock initialized: now 1790507291039708 us; error 35 us; skew 500 ppm
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507291, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:11.042089 20408 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:11.042523 20408 webserver.cc:460] Webserver started at http://127.17.95.194:41931/ using document root <none> and password file <none>
I20260927 11:08:11.042734 20408 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:11.042799 20408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:11.042943 20408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:11.044030 20408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "6b0022c0eebc4b838be3b74e25813a51"
format_stamp: "Formatted at 2026-09-27 11:08:11 on dist-test-slave-6j0v"
server_key: "cc09d38a24f4def86a9086ae8a60d591"
server_key_iv: "fbab4fee2e88ccd0e33c977be1f096b1"
server_key_version: "encryptionkey@0"
I20260927 11:08:11.044379 20408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "6b0022c0eebc4b838be3b74e25813a51"
format_stamp: "Formatted at 2026-09-27 11:08:11 on dist-test-slave-6j0v"
server_key: "cc09d38a24f4def86a9086ae8a60d591"
server_key_iv: "fbab4fee2e88ccd0e33c977be1f096b1"
server_key_version: "encryptionkey@0"
I20260927 11:08:11.045673 20408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:11.046382 20423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.046552 20408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:11.046646 20408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "6b0022c0eebc4b838be3b74e25813a51"
format_stamp: "Formatted at 2026-09-27 11:08:11 on dist-test-slave-6j0v"
server_key: "cc09d38a24f4def86a9086ae8a60d591"
server_key_iv: "fbab4fee2e88ccd0e33c977be1f096b1"
server_key_version: "encryptionkey@0"
I20260927 11:08:11.046727 20408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:11.055907 20408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:11.057719 20408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:11.057884 20408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:11.058117 20408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:11.058413 20408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:11.058480 20408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.058521 20408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:11.058552 20408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.064224 20408 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:40427
I20260927 11:08:11.064270 20536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:40427 every 8 connection(s)
I20260927 11:08:11.064556 20408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:11.064919 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20408
I20260927 11:08:11.065016 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:11.065209 17791 external_mini_cluster.cc:1428] Setting key e623f9a00edef4d240baac84a04affbb
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507291, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
I20260927 11:08:11.074447 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:11.065199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:59315 (local address 127.17.95.254:42459)
0927 11:08:11.065253 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:11.065255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:11.065481 (+ 226us) server_negotiation.cc:407] Connection header received
0927 11:08:11.066078 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:11.066079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:11.066091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:11.066149 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:11.067268 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.067762 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.068759 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.069080 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.069149 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:11.071355 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:11.071357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:11.071357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:11.071366 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:11.073005 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:11.073377 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:11.073378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:11.073379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:11.073442 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:11.073674 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:11.073675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:11.073676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:11.073825 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:11.073908 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:11.074226 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 11:08:11.074318 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 11:08:11.074615 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:11.065198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42459 (local address 127.17.95.194:59315)
0927 11:08:11.065350 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:11.065388 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:11.065975 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:11.066218 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:11.066223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:11.066772 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:11.067154 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:11.067160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.067856 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.067857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:11.068656 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:11.068658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.069167 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.069168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:11.069274 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:11.069788 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:11.069795 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:11.071263 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:11.073091 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:11.073096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:11.073098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:11.073299 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:11.073522 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:11.073523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:11.073523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:11.073610 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:11.073930 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:11.073951 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:11.074077 (+ 126us) client_negotiation.cc:769] Sending connection context
0927 11:08:11.074188 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 11:08:11.074322 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260927 11:08:11.075503 20537 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42459
I20260927 11:08:11.075616 20537 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:11.075834 20537 heartbeater.cc:511] Master 127.17.95.254:42459 requested a full tablet report, sending...
I20260927 11:08:11.076282 20212 ts_manager.cc:194] Registered new tserver with Master: 6b0022c0eebc4b838be3b74e25813a51 (127.17.95.194:40427)
I20260927 11:08:11.076613 20212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:59315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:11.089970 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42459
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:11.168437 20542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:11.168612 20542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:11.168632 20542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:11.169651 20542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:11.169689 20542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:11.169713 20542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:11.171043 20542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:11.171291 20542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:11.171497 20542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:11.175923 20548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:11.175982 20550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:11.175922 20547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:11.176237 20542 server_base.cc:1034] running on GCE node
I20260927 11:08:11.176501 20542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:11.176741 20542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:11.177913 20542 hybrid_clock.cc:648] HybridClock initialized: now 1790507291177905 us; error 31 us; skew 500 ppm
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507291, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:11.180380 20542 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:11.180850 20542 webserver.cc:460] Webserver started at http://127.17.95.195:41065/ using document root <none> and password file <none>
I20260927 11:08:11.181097 20542 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:11.181182 20542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:11.181339 20542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:11.182325 20542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "c723fac7b4b84a6d93b7d8343d3f09d2"
format_stamp: "Formatted at 2026-09-27 11:08:11 on dist-test-slave-6j0v"
server_key: "19bd54b8dbddb567e6665de0ad5511dc"
server_key_iv: "fcff11e80cd9b76ee59c1bd7efaa0a3a"
server_key_version: "encryptionkey@0"
I20260927 11:08:11.182703 20542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "c723fac7b4b84a6d93b7d8343d3f09d2"
format_stamp: "Formatted at 2026-09-27 11:08:11 on dist-test-slave-6j0v"
server_key: "19bd54b8dbddb567e6665de0ad5511dc"
server_key_iv: "fcff11e80cd9b76ee59c1bd7efaa0a3a"
server_key_version: "encryptionkey@0"
I20260927 11:08:11.184047 20542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:11.184773 20557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.184967 20542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.185071 20542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "c723fac7b4b84a6d93b7d8343d3f09d2"
format_stamp: "Formatted at 2026-09-27 11:08:11 on dist-test-slave-6j0v"
server_key: "19bd54b8dbddb567e6665de0ad5511dc"
server_key_iv: "fcff11e80cd9b76ee59c1bd7efaa0a3a"
server_key_version: "encryptionkey@0"
I20260927 11:08:11.185151 20542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:11.220818 20542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:11.222636 20542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:11.222793 20542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:11.223022 20542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:11.223363 20542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:11.223419 20542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.223467 20542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:11.223498 20542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:11.229755 20542 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:43055
I20260927 11:08:11.229807 20670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:43055 every 8 connection(s)
I20260927 11:08:11.230188 20542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:11.234915 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20542
I20260927 11:08:11.235044 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:11.235335 17791 external_mini_cluster.cc:1428] Setting key 33977e92f1f79f4dcc4c77ca877f3bf6
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507291, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:11.239583 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:11.230888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:56773 (local address 127.17.95.254:42459)
0927 11:08:11.230930 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:11.230932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:11.231153 (+ 221us) server_negotiation.cc:407] Connection header received
0927 11:08:11.231618 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:11.231619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:11.231630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:11.231691 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:11.232574 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.233275 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.234375 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.234603 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.234666 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:11.237090 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:11.237091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:11.237091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:11.237097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:11.238287 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:11.238632 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:11.238633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:11.238633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:11.238684 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:11.238989 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:11.238989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:11.238990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:11.239139 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:11.239204 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:11.239458 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 11:08:11.239518 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:11.239872 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:11.230859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42459 (local address 127.17.95.195:56773)
0927 11:08:11.231013 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:11.231053 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:11.231531 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:11.231757 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:11.231761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:11.232294 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:11.232468 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:11.232473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.233372 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.233372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:11.234141 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:11.234143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:11.234686 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:11.234687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:11.234791 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:11.235418 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:11.235427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:11.237015 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:11.238355 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:11.238360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:11.238361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:11.238565 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:11.238775 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:11.238776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:11.238777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:11.238903 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:11.239206 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:11.239212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:11.239328 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 11:08:11.239426 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 11:08:11.239566 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260927 11:08:11.240386 20671 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42459
I20260927 11:08:11.240496 20671 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:11.240646 20671 heartbeater.cc:511] Master 127.17.95.254:42459 requested a full tablet report, sending...
I20260927 11:08:11.240978 20212 ts_manager.cc:194] Registered new tserver with Master: c723fac7b4b84a6d93b7d8343d3f09d2 (127.17.95.195:43055)
I20260927 11:08:11.241374 20212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:56773
I20260927 11:08:11.247493 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 11:08:11.258948 20255 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:11.258122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.17.95.254:42459)
0927 11:08:11.258214 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:11.258216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:11.258251 (+ 35us) server_negotiation.cc:407] Connection header received
0927 11:08:11.258330 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:11.258331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:11.258354 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:11.258401 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:11.258880 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46780: BlockingRecv error: recv got EOF from 127.0.0.1:46780 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 11:08:11.259758 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20274
I20260927 11:08:11.263926 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20408
I20260927 11:08:11.268260 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20542
I20260927 11:08:11.272156 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20180
2026-09-27T11:08:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1809 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20683](info): set up 2 sockets
Sep 27 11:08:11 dist-test-slave-6j0v krb5kdc[20683](info): commencing operation
krb5kdc: starting...
W20260927 11:08:12.274992 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:12 dist-test-slave-6j0v krb5kdc[20683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:12.385835 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:41891
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:41891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:12.461790 20696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:12.462009 20696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:12.462044 20696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:12.463302 20696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:12.463352 20696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:12.463374 20696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:12.463400 20696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:12.463418 20696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:12.465140 20696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:41891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:41891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:12.465404 20696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:12.465629 20696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:12.470328 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:12.470352 20704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:12.470328 20702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:12.470710 20696 server_base.cc:1034] running on GCE node
I20260927 11:08:12.470976 20696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:12.471313 20696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:12.472451 20696 hybrid_clock.cc:648] HybridClock initialized: now 1790507292472428 us; error 38 us; skew 500 ppm
Sep 27 11:08:12 dist-test-slave-6j0v krb5kdc[20683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507292, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:12.474836 20696 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:12.475296 20696 webserver.cc:460] Webserver started at http://127.17.95.254:34675/ using document root <none> and password file <none>
I20260927 11:08:12.475534 20696 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:12.475600 20696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:12.475728 20696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:12.476716 20696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "584a03bc32774bcc83b5680a3e9b3c4d"
format_stamp: "Formatted at 2026-09-27 11:08:12 on dist-test-slave-6j0v"
server_key: "95957227024ecf340355c0038eeb9b81"
server_key_iv: "6f922ff84eb57476f6eb58a5759d067e"
server_key_version: "encryptionkey@0"
I20260927 11:08:12.477093 20696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "584a03bc32774bcc83b5680a3e9b3c4d"
format_stamp: "Formatted at 2026-09-27 11:08:12 on dist-test-slave-6j0v"
server_key: "95957227024ecf340355c0038eeb9b81"
server_key_iv: "6f922ff84eb57476f6eb58a5759d067e"
server_key_version: "encryptionkey@0"
I20260927 11:08:12.478341 20696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:12.479002 20711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:12.479246 20696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:12.479341 20696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "584a03bc32774bcc83b5680a3e9b3c4d"
format_stamp: "Formatted at 2026-09-27 11:08:12 on dist-test-slave-6j0v"
server_key: "95957227024ecf340355c0038eeb9b81"
server_key_iv: "6f922ff84eb57476f6eb58a5759d067e"
server_key_version: "encryptionkey@0"
I20260927 11:08:12.479413 20696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:12.488557 20696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:12.490463 20696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:12.490621 20696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:12.494746 20696 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:41891
I20260927 11:08:12.494797 20763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:41891 every 8 connection(s)
I20260927 11:08:12.495357 20696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:12.496022 20764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:12.498337 20764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d: Bootstrap starting.
I20260927 11:08:12.499195 20764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:12.499536 20764 log.cc:824] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:12.500013 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20696
I20260927 11:08:12.500111 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:12.500317 17791 external_mini_cluster.cc:1428] Setting key bfbf580d2864e51e297fea29a4c1b1ab
I20260927 11:08:12.500358 20764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d: No bootstrap required, opened a new log
I20260927 11:08:12.503422 20764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41891 } }
I20260927 11:08:12.503594 20764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:12.503631 20764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584a03bc32774bcc83b5680a3e9b3c4d, State: Initialized, Role: FOLLOWER
I20260927 11:08:12.503793 20764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41891 } }
I20260927 11:08:12.503876 20764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:12.503918 20764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:12.503973 20764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:12.504675 20764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41891 } }
I20260927 11:08:12.504791 20764 leader_election.cc:302] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 584a03bc32774bcc83b5680a3e9b3c4d; no voters:
I20260927 11:08:12.504980 20764 leader_election.cc:288] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:12.505016 20769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:12.505210 20764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:12.505231 20769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [term 1 LEADER]: Becoming Leader. State: Replica: 584a03bc32774bcc83b5680a3e9b3c4d, State: Running, Role: LEADER
I20260927 11:08:12.505347 20769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41891 } }
I20260927 11:08:12.507099 20769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 584a03bc32774bcc83b5680a3e9b3c4d. Latest consensus state: current_term: 1 leader_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41891 } } }
I20260927 11:08:12.507179 20769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:12.507623 20770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584a03bc32774bcc83b5680a3e9b3c4d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41891 } } }
I20260927 11:08:12.507686 20770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d [sys.catalog]: This master's current role is: LEADER
W20260927 11:08:12.508513 20784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:08:12.508566 20784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:08:12.508677 20783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:12.508929 20783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 11:08:12 dist-test-slave-6j0v krb5kdc[20683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:12.511971 20783 catalog_manager.cc:1347] Generated new cluster ID: 8530a564f12740de9dd3d21a7d32f83d
I20260927 11:08:12.512023 20783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:12.513429 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:12.500926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.17.95.254:41891)
0927 11:08:12.501377 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:12.501386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:12.501416 (+ 30us) server_negotiation.cc:407] Connection header received
0927 11:08:12.502316 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:12.502325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:12.502384 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:12.502515 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:12.503397 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:12.505460 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:12.506616 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:12.507253 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:12.507660 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:12.510162 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:12.510166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:12.510166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:12.510178 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:12.511809 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:12.512174 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:12.512177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:12.512177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:12.512287 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:12.512493 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:12.512494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:12.512494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:12.512797 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:12.512947 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:12.513108 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 11:08:12.513231 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":79,"threads_started":1}
I20260927 11:08:12.514367 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:08:12.520788 20783 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:12.521518 20783 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:12.529917 20783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 584a03bc32774bcc83b5680a3e9b3c4d: Generated new TSK 0
I20260927 11:08:12.530069 20783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:12.533181 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:12.529749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.17.95.254:41891)
0927 11:08:12.529790 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:12.529792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:12.529975 (+ 183us) server_negotiation.cc:407] Connection header received
0927 11:08:12.530080 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:12.530081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:12.530092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:12.530143 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:12.530503 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:12.530986 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:12.531758 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:12.531959 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:12.532040 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:12.532732 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:12.532733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:12.532733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:12.532894 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:12.532907 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:12.532913 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:12.532974 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:12.533061 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 11:08:12.533083 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:12.535315 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20696
2026-09-27T11:08:12Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1259 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:12 dist-test-slave-6j0v krb5kdc[20797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:12 dist-test-slave-6j0v krb5kdc[20797](info): set up 2 sockets
Sep 27 11:08:12 dist-test-slave-6j0v krb5kdc[20797](info): commencing operation
krb5kdc: starting...
W20260927 11:08:13.542966 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:13 dist-test-slave-6j0v krb5kdc[20797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:13.656138 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34093
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:13.734278 20810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:13.734458 20810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:13.734479 20810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:13.735644 20810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:13.735682 20810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:13.735695 20810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:13.735706 20810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:13.735718 20810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:13.737142 20810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:13.737363 20810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:13.737583 20810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:13.742174 20815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:13.742143 20816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:13.742151 20818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:13.742575 20810 server_base.cc:1034] running on GCE node
I20260927 11:08:13.742813 20810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:13.743117 20810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:13.744282 20810 hybrid_clock.cc:648] HybridClock initialized: now 1790507293744254 us; error 34 us; skew 500 ppm
Sep 27 11:08:13 dist-test-slave-6j0v krb5kdc[20797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:13.751477 20810 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:13.751952 20810 webserver.cc:460] Webserver started at http://127.17.95.254:41129/ using document root <none> and password file <none>
I20260927 11:08:13.752190 20810 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:13.752261 20810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:13.752389 20810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:13.753456 20810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "72a43da9d8ca418faf3d7a0a7b1a9565"
format_stamp: "Formatted at 2026-09-27 11:08:13 on dist-test-slave-6j0v"
server_key: "06adae644c7b9fb009b12be089de71ec"
server_key_iv: "44c0610b5c04d9605b82f91ea162211f"
server_key_version: "encryptionkey@0"
I20260927 11:08:13.753813 20810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "72a43da9d8ca418faf3d7a0a7b1a9565"
format_stamp: "Formatted at 2026-09-27 11:08:13 on dist-test-slave-6j0v"
server_key: "06adae644c7b9fb009b12be089de71ec"
server_key_iv: "44c0610b5c04d9605b82f91ea162211f"
server_key_version: "encryptionkey@0"
I20260927 11:08:13.755054 20810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:08:13.755820 20825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:13.756016 20810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:13.756101 20810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "72a43da9d8ca418faf3d7a0a7b1a9565"
format_stamp: "Formatted at 2026-09-27 11:08:13 on dist-test-slave-6j0v"
server_key: "06adae644c7b9fb009b12be089de71ec"
server_key_iv: "44c0610b5c04d9605b82f91ea162211f"
server_key_version: "encryptionkey@0"
I20260927 11:08:13.756152 20810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:13.772045 20810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:13.774294 20810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:13.774408 20810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:13.778142 20810 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:34093
I20260927 11:08:13.778172 20877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:34093 every 8 connection(s)
I20260927 11:08:13.778512 20810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:13.779501 20878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:13.780296 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20810
I20260927 11:08:13.780395 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:13.780567 17791 external_mini_cluster.cc:1428] Setting key 2c87844e6651b59a239b01caa3f45bc6
I20260927 11:08:13.782418 20878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565: Bootstrap starting.
I20260927 11:08:13.783048 20878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:13.783401 20878 log.cc:824] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:13.784051 20878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565: No bootstrap required, opened a new log
I20260927 11:08:13.785766 20878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34093 } }
I20260927 11:08:13.785890 20878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:13.785912 20878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a43da9d8ca418faf3d7a0a7b1a9565, State: Initialized, Role: FOLLOWER
I20260927 11:08:13.786046 20878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34093 } }
I20260927 11:08:13.786115 20878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:13.786136 20878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:13.786201 20878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 11:08:13 dist-test-slave-6j0v krb5kdc[20797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:13.786751 20878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34093 } }
I20260927 11:08:13.786865 20878 leader_election.cc:302] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a43da9d8ca418faf3d7a0a7b1a9565; no voters:
I20260927 11:08:13.787060 20878 leader_election.cc:288] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:13.787111 20883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:13.787257 20883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [term 1 LEADER]: Becoming Leader. State: Replica: 72a43da9d8ca418faf3d7a0a7b1a9565, State: Running, Role: LEADER
I20260927 11:08:13.787335 20883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34093 } }
I20260927 11:08:13.787374 20878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:13.787618 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a43da9d8ca418faf3d7a0a7b1a9565. Latest consensus state: current_term: 1 leader_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34093 } } }
I20260927 11:08:13.787686 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:13.788007 20885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a43da9d8ca418faf3d7a0a7b1a9565" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34093 } } }
I20260927 11:08:13.788064 20885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:13.788225 20887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:13.788694 20887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:13.789945 20900 mvcc.cc:205] Tried to move back new op lower bound from 7333917875361779712 to 7333917875353276416. Current Snapshot: MvccSnapshot[applied={T|T < 7333917875361779712}]
I20260927 11:08:13.790194 20887 catalog_manager.cc:1347] Generated new cluster ID: e7ad8dc48e7c46d68b03e1bdfc302501
I20260927 11:08:13.790233 20887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:13.791216 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:13.781031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.17.95.254:34093)
0927 11:08:13.781186 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:13.781197 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:13.781220 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:08:13.781601 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:13.781607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:13.781647 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:13.781755 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:13.782472 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:13.783577 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:13.784365 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:13.784754 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:13.784875 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:13.787371 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:13.787375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:13.787375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:13.787386 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:13.789600 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:13.790004 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:13.790006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:13.790006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:13.790101 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:13.790301 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:13.790301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:13.790302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:13.790534 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:13.790618 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:13.790857 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 11:08:13.790990 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260927 11:08:13.810619 20887 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:13.811182 20887 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:13.815543 20887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a43da9d8ca418faf3d7a0a7b1a9565: Generated new TSK 0
I20260927 11:08:13.815732 20887 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:13.840189 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34093
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:13.917091 20904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:13.917301 20904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:13.917353 20904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:13.918445 20904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:13.918514 20904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:13.918557 20904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:13.919934 20904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:13.920202 20904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:13.920420 20904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:13.925217 20909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:13.925297 20912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:13.925307 20910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:13.925570 20904 server_base.cc:1034] running on GCE node
I20260927 11:08:13.925791 20904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:13.926106 20904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:13.927281 20904 hybrid_clock.cc:648] HybridClock initialized: now 1790507293927260 us; error 33 us; skew 500 ppm
Sep 27 11:08:13 dist-test-slave-6j0v krb5kdc[20797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:13.929828 20904 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:13.930244 20904 webserver.cc:460] Webserver started at http://127.17.95.193:38013/ using document root <none> and password file <none>
I20260927 11:08:13.930495 20904 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:13.930565 20904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:13.930713 20904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:13.931846 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "68bb2c2c29f74533aaf59602e130c7e4"
format_stamp: "Formatted at 2026-09-27 11:08:13 on dist-test-slave-6j0v"
server_key: "84797b8f51504c94339be3e9241390de"
server_key_iv: "2d6a8b900f1cc979d435b0599e4de047"
server_key_version: "encryptionkey@0"
I20260927 11:08:13.932222 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "68bb2c2c29f74533aaf59602e130c7e4"
format_stamp: "Formatted at 2026-09-27 11:08:13 on dist-test-slave-6j0v"
server_key: "84797b8f51504c94339be3e9241390de"
server_key_iv: "2d6a8b900f1cc979d435b0599e4de047"
server_key_version: "encryptionkey@0"
I20260927 11:08:13.933666 20904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:13.934505 20919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:13.934674 20904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:13.934772 20904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "68bb2c2c29f74533aaf59602e130c7e4"
format_stamp: "Formatted at 2026-09-27 11:08:13 on dist-test-slave-6j0v"
server_key: "84797b8f51504c94339be3e9241390de"
server_key_iv: "2d6a8b900f1cc979d435b0599e4de047"
server_key_version: "encryptionkey@0"
I20260927 11:08:13.934855 20904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:13.947252 20904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:13.949107 20904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:13.949262 20904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:13.949496 20904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:13.949807 20904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:13.949863 20904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:13.949904 20904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:13.949934 20904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:13.956277 20904 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:44963
I20260927 11:08:13.956364 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:44963 every 8 connection(s)
I20260927 11:08:13.956676 20904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:13 dist-test-slave-6j0v krb5kdc[20797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507293, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:13.964641 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 20904
I20260927 11:08:13.964717 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:13.964927 17791 external_mini_cluster.cc:1428] Setting key ae5351a57b7a66be19b1c9c30e39baf4
I20260927 11:08:13.966320 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:13.957405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:51609 (local address 127.17.95.254:34093)
0927 11:08:13.957466 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:13.957468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:13.957702 (+ 234us) server_negotiation.cc:407] Connection header received
0927 11:08:13.958245 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:13.958247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:13.958258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:13.958323 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:13.959390 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:13.960051 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:13.961146 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:13.961380 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:13.961448 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:13.963368 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:13.963369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:13.963369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:13.963376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:13.964764 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:13.965229 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:13.965230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:13.965230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:13.965309 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:13.965684 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:13.965685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:13.965685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:13.965810 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:13.965888 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:13.966142 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 11:08:13.966229 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 11:08:13.966502 21035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:13.957384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34093 (local address 127.17.95.193:51609)
0927 11:08:13.957546 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:13.957612 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:13.958125 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:13.958377 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:13.958380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:13.959040 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:13.959308 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:13.959313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:13.960204 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:13.960206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:13.961039 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:13.961041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:13.961475 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:13.961475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:13.961571 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:13.961965 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:13.961973 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:13.963291 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:13.964830 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:13.964837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:13.964840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:13.965140 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:13.965402 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:13.965403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:13.965403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:13.965600 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:13.965878 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:13.965884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:13.966007 (+ 123us) client_negotiation.cc:769] Sending connection context
0927 11:08:13.966107 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 11:08:13.966261 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
I20260927 11:08:13.967082 21033 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34093
I20260927 11:08:13.967176 21033 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:13.967372 21033 heartbeater.cc:511] Master 127.17.95.254:34093 requested a full tablet report, sending...
I20260927 11:08:13.967854 20842 ts_manager.cc:194] Registered new tserver with Master: 68bb2c2c29f74533aaf59602e130c7e4 (127.17.95.193:44963)
I20260927 11:08:13.968422 20842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:51609
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:13.989424 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34093
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:14.068390 21038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:14.068584 21038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:14.068608 21038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:14.069737 21038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:14.069795 21038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:14.069841 21038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:14.071259 21038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:14.071482 21038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:14.071722 21038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:14.076776 21046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:14.076730 21043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:14.076741 21044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:14.076712 21038 server_base.cc:1034] running on GCE node
I20260927 11:08:14.077096 21038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:14.077291 21038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:14.078449 21038 hybrid_clock.cc:648] HybridClock initialized: now 1790507294078424 us; error 52 us; skew 500 ppm
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[20797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507294, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:14.080865 21038 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:14.081279 21038 webserver.cc:460] Webserver started at http://127.17.95.194:38475/ using document root <none> and password file <none>
I20260927 11:08:14.081506 21038 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:14.081570 21038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:14.081712 21038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:14.082684 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "f8fd2278e4b24ceda5800c250811b882"
format_stamp: "Formatted at 2026-09-27 11:08:14 on dist-test-slave-6j0v"
server_key: "3d0d900214950f96eab6fca7bafb01e1"
server_key_iv: "e981aeeead7670c0ad16ceebf4e34b1d"
server_key_version: "encryptionkey@0"
I20260927 11:08:14.083025 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "f8fd2278e4b24ceda5800c250811b882"
format_stamp: "Formatted at 2026-09-27 11:08:14 on dist-test-slave-6j0v"
server_key: "3d0d900214950f96eab6fca7bafb01e1"
server_key_iv: "e981aeeead7670c0ad16ceebf4e34b1d"
server_key_version: "encryptionkey@0"
I20260927 11:08:14.084388 21038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:14.085187 21053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:14.085357 21038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:14.085455 21038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "f8fd2278e4b24ceda5800c250811b882"
format_stamp: "Formatted at 2026-09-27 11:08:14 on dist-test-slave-6j0v"
server_key: "3d0d900214950f96eab6fca7bafb01e1"
server_key_iv: "e981aeeead7670c0ad16ceebf4e34b1d"
server_key_version: "encryptionkey@0"
I20260927 11:08:14.085532 21038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:14.096931 21038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:14.098613 21038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:14.098793 21038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:14.098989 21038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:14.099287 21038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:14.099337 21038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:14.099370 21038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:14.099395 21038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:14.104799 21038 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:41641
I20260927 11:08:14.104835 21166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:41641 every 8 connection(s)
I20260927 11:08:14.105146 21038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[20797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507294, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:14.113936 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21038
I20260927 11:08:14.114051 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:14.114261 17791 external_mini_cluster.cc:1428] Setting key 1727ba283ebf25bcc09cd68d90d12bcb
I20260927 11:08:14.114912 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:14.105876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:60159 (local address 127.17.95.254:34093)
0927 11:08:14.105918 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:14.105920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:14.106151 (+ 231us) server_negotiation.cc:407] Connection header received
0927 11:08:14.106618 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:14.106620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:14.106630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:14.106678 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:14.108094 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.108579 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.109595 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.109764 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.109812 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:14.111975 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:14.111981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:14.111981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:14.111988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:14.113222 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:14.113656 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:14.113658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:14.113658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:14.113875 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:14.114233 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:14.114234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:14.114235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:14.114374 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:14.114448 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:14.114729 (+ 281us) server_negotiation.cc:299] Negotiation successful
0927 11:08:14.114819 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:14.115146 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:14.105846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34093 (local address 127.17.95.194:60159)
0927 11:08:14.106009 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:14.106050 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:14.106535 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:14.106735 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:14.106738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:14.107828 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:14.108015 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:14.108020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.108671 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.108672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:14.109505 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:14.109507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.109845 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.109846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:14.109947 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:14.110422 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:14.110429 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:14.111849 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:14.113310 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:14.113315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:14.113317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:14.113563 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:14.113969 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:14.113970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:14.113970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:14.114154 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:14.114448 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:14.114455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:14.114611 (+ 156us) client_negotiation.cc:769] Sending connection context
0927 11:08:14.114694 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 11:08:14.114810 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260927 11:08:14.115648 21167 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34093
I20260927 11:08:14.115727 21167 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:14.115912 21167 heartbeater.cc:511] Master 127.17.95.254:34093 requested a full tablet report, sending...
I20260927 11:08:14.116285 20842 ts_manager.cc:194] Registered new tserver with Master: f8fd2278e4b24ceda5800c250811b882 (127.17.95.194:41641)
I20260927 11:08:14.116744 20842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:60159
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:14.150158 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34093
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:14.226485 21172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:14.226653 21172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:14.226672 21172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:14.227859 21172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:14.227898 21172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:14.227924 21172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:14.229301 21172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:14.229509 21172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:14.229746 21172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:14.234265 21177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:14.234249 21180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:14.234411 21178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:14.234611 21172 server_base.cc:1034] running on GCE node
I20260927 11:08:14.234802 21172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:14.235052 21172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:14.236222 21172 hybrid_clock.cc:648] HybridClock initialized: now 1790507294236222 us; error 45 us; skew 500 ppm
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[20797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507294, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:14.238715 21172 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:14.239061 21172 webserver.cc:460] Webserver started at http://127.17.95.195:41635/ using document root <none> and password file <none>
I20260927 11:08:14.239306 21172 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:14.239347 21172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:14.239440 21172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:14.240376 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "70c90e506a4141548b1f0038a2995d5e"
format_stamp: "Formatted at 2026-09-27 11:08:14 on dist-test-slave-6j0v"
server_key: "21e29c024d0bcc44c0935b5d1b3c9436"
server_key_iv: "c9735311d37d95b27232dfd3a857d479"
server_key_version: "encryptionkey@0"
I20260927 11:08:14.240655 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "70c90e506a4141548b1f0038a2995d5e"
format_stamp: "Formatted at 2026-09-27 11:08:14 on dist-test-slave-6j0v"
server_key: "21e29c024d0bcc44c0935b5d1b3c9436"
server_key_iv: "c9735311d37d95b27232dfd3a857d479"
server_key_version: "encryptionkey@0"
I20260927 11:08:14.241816 21172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:08:14.242504 21187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:14.242690 21172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:14.242777 21172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "70c90e506a4141548b1f0038a2995d5e"
format_stamp: "Formatted at 2026-09-27 11:08:14 on dist-test-slave-6j0v"
server_key: "21e29c024d0bcc44c0935b5d1b3c9436"
server_key_iv: "c9735311d37d95b27232dfd3a857d479"
server_key_version: "encryptionkey@0"
I20260927 11:08:14.242853 21172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:14.254531 21172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:14.256379 21172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:14.256536 21172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:14.256731 21172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:14.257014 21172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:14.257067 21172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:14.257104 21172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:14.257133 21172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:14.262806 21172 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:45305
I20260927 11:08:14.262850 21300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:45305 every 8 connection(s)
I20260927 11:08:14.263199 21172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:14.264349 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21172
I20260927 11:08:14.264487 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:14.264688 17791 external_mini_cluster.cc:1428] Setting key 0bc8b6286721e66eeab971773116be1c
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[20797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507294, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:14.272054 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:14.263842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:48053 (local address 127.17.95.254:34093)
0927 11:08:14.263926 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:14.263928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:14.264090 (+ 162us) server_negotiation.cc:407] Connection header received
0927 11:08:14.264650 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:14.264651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:14.264675 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:14.264733 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:14.265877 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.266413 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.267424 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.267636 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.267720 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:14.269657 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:14.269658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:14.269658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:14.269664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:14.271021 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:14.271379 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:14.271380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:14.271380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:14.271430 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:14.271618 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:14.271618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:14.271619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:14.271702 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:14.271744 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:14.271920 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 11:08:14.271977 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 11:08:14.272255 21303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:14.263772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34093 (local address 127.17.95.195:48053)
0927 11:08:14.263959 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:14.263999 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:14.264521 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:14.264799 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:14.264803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:14.265582 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:14.265782 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:14.265786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.266505 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.266506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:14.267334 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:14.267336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.267727 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.267728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:14.267850 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:14.268243 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:14.268250 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:14.269580 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:14.271131 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:14.271140 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:14.271142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:14.271329 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:14.271488 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:14.271489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:14.271489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:14.271582 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:14.271746 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:14.271750 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:14.271839 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 11:08:14.271908 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:08:14.272031 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260927 11:08:14.272760 21301 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34093
I20260927 11:08:14.272863 21301 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:14.273029 21301 heartbeater.cc:511] Master 127.17.95.254:34093 requested a full tablet report, sending...
I20260927 11:08:14.273370 20842 ts_manager.cc:194] Registered new tserver with Master: 70c90e506a4141548b1f0038a2995d5e (127.17.95.195:45305)
I20260927 11:08:14.273705 20842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:48053
I20260927 11:08:14.276489 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[20797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507294, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[20797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507294, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:14.297845 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:14.291254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.17.95.254:34093)
0927 11:08:14.291319 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:14.291321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:14.291440 (+ 119us) server_negotiation.cc:407] Connection header received
0927 11:08:14.291493 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:14.291494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:14.291503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:14.291560 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:14.292076 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.292529 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.293186 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:14.293349 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:14.293395 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:14.295614 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:14.295616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:14.295616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:14.295622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:14.296766 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:14.297066 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:14.297067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:14.297067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:14.297134 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:14.297269 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:14.297269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:14.297270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:14.297440 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:14.297531 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:14.297674 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 11:08:14.297746 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260927 11:08:14.298187 20842 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53390
I20260927 11:08:14.298693 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20904
I20260927 11:08:14.304122 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21038
I20260927 11:08:14.308935 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21172
I20260927 11:08:14.313669 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 20810
2026-09-27T11:08:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1780 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[21313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[21313](info): set up 2 sockets
Sep 27 11:08:14 dist-test-slave-6j0v krb5kdc[21313](info): commencing operation
krb5kdc: starting...
W20260927 11:08:15.374886 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:15.487674 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39459
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:39459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:15.567021 21326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:15.567250 21326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:15.567274 21326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:15.568490 21326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:15.568534 21326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:15.568552 21326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:15.568565 21326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:15.568619 21326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:15.570068 21326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:39459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:15.570312 21326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:15.570567 21326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:15.575255 21331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:15.575286 21334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:15.575363 21326 server_base.cc:1034] running on GCE node
W20260927 11:08:15.575248 21332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:15.575726 21326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:15.575999 21326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:15.577157 21326 hybrid_clock.cc:648] HybridClock initialized: now 1790507295577139 us; error 42 us; skew 500 ppm
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:15.579773 21326 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:15.580171 21326 webserver.cc:460] Webserver started at http://127.17.95.254:39005/ using document root <none> and password file <none>
I20260927 11:08:15.580358 21326 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:15.580399 21326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:15.580509 21326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:15.581621 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "089fd21189dd436dbe5edc024453ad78"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "24f391751684abafb4c4d7c8fed1ce93"
server_key_iv: "fcdeb93579973fad000981a8d257eb4e"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.581931 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "089fd21189dd436dbe5edc024453ad78"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "24f391751684abafb4c4d7c8fed1ce93"
server_key_iv: "fcdeb93579973fad000981a8d257eb4e"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.583201 21326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:15.584028 21341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.584213 21326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:15.584296 21326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "089fd21189dd436dbe5edc024453ad78"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "24f391751684abafb4c4d7c8fed1ce93"
server_key_iv: "fcdeb93579973fad000981a8d257eb4e"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.584353 21326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:15.594421 21326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:15.596275 21326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:15.596379 21326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:15.600097 21326 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:39459
I20260927 11:08:15.600126 21393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:39459 every 8 connection(s)
I20260927 11:08:15.600684 21326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:15.601317 21394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:15.601766 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21326
I20260927 11:08:15.601845 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:15.602058 17791 external_mini_cluster.cc:1428] Setting key 0ed9bb5f3cae81859eeefde2d4fbe4b9
I20260927 11:08:15.604215 21394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78: Bootstrap starting.
I20260927 11:08:15.604796 21394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:15.605070 21394 log.cc:824] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:15.605719 21394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78: No bootstrap required, opened a new log
I20260927 11:08:15.607750 21394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd21189dd436dbe5edc024453ad78" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39459 } }
I20260927 11:08:15.607895 21394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:15.607944 21394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089fd21189dd436dbe5edc024453ad78, State: Initialized, Role: FOLLOWER
I20260927 11:08:15.608086 21394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd21189dd436dbe5edc024453ad78" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39459 } }
I20260927 11:08:15.608182 21394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:15.608222 21394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:15.608268 21394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:15.608901 21394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd21189dd436dbe5edc024453ad78" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39459 } }
I20260927 11:08:15.609021 21394 leader_election.cc:302] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089fd21189dd436dbe5edc024453ad78; no voters:
I20260927 11:08:15.609195 21394 leader_election.cc:288] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:15.609315 21399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:15.609440 21394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:15.609529 21399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [term 1 LEADER]: Becoming Leader. State: Replica: 089fd21189dd436dbe5edc024453ad78, State: Running, Role: LEADER
I20260927 11:08:15.609684 21399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd21189dd436dbe5edc024453ad78" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39459 } }
I20260927 11:08:15.610024 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089fd21189dd436dbe5edc024453ad78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd21189dd436dbe5edc024453ad78" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39459 } } }
I20260927 11:08:15.610102 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:15.610246 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089fd21189dd436dbe5edc024453ad78. Latest consensus state: current_term: 1 leader_uuid: "089fd21189dd436dbe5edc024453ad78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd21189dd436dbe5edc024453ad78" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39459 } } }
I20260927 11:08:15.610291 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:15.610301 21408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:15.610859 21408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:15.612529 21408 catalog_manager.cc:1347] Generated new cluster ID: 8f0495b29aa84be8ac2d83c175bba61c
I20260927 11:08:15.612576 21408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:15.613286 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:15.602554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.17.95.254:39459)
0927 11:08:15.602721 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:15.602728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:15.602752 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:08:15.603206 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:15.603212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:15.603257 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:15.603369 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:15.604121 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.605097 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.605851 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.606252 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.606396 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:15.608782 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:15.608786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:15.608786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:15.608794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:15.611668 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:15.612086 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:15.612088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:15.612088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:15.612176 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:15.612376 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:15.612377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:15.612377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:15.612687 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:15.612813 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:15.612986 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 11:08:15.613104 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260927 11:08:15.616978 21408 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:15.617412 21408 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:15.623632 21408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 089fd21189dd436dbe5edc024453ad78: Generated new TSK 0
I20260927 11:08:15.623777 21408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:15.654111 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39459
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:15.732774 21420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:15.732966 21420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:15.732990 21420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:15.734123 21420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:15.734161 21420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:15.734184 21420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:15.735664 21420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:15.735893 21420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:15.736122 21420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:15.740883 21426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:15.740868 21425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:15.740865 21428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:15.740902 21420 server_base.cc:1034] running on GCE node
I20260927 11:08:15.741206 21420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:15.741460 21420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:15.742633 21420 hybrid_clock.cc:648] HybridClock initialized: now 1790507295742608 us; error 36 us; skew 500 ppm
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:15.745371 21420 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:15.745817 21420 webserver.cc:460] Webserver started at http://127.17.95.193:40485/ using document root <none> and password file <none>
I20260927 11:08:15.746053 21420 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:15.746100 21420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:15.746282 21420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:15.747516 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "1c6e31d20b844c45b204e367f9c8f7cb"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "e17e78e024692edf621d5905553ed81b"
server_key_iv: "088f46eda33d6c74e951b8d64087fc1a"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.747915 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "1c6e31d20b844c45b204e367f9c8f7cb"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "e17e78e024692edf621d5905553ed81b"
server_key_iv: "088f46eda33d6c74e951b8d64087fc1a"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.749291 21420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:15.750067 21435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.750250 21420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:15.750345 21420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "1c6e31d20b844c45b204e367f9c8f7cb"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "e17e78e024692edf621d5905553ed81b"
server_key_iv: "088f46eda33d6c74e951b8d64087fc1a"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.750419 21420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:15.772516 21420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:15.775022 21420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:15.775220 21420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:15.775503 21420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:15.775822 21420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:15.775882 21420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.775929 21420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:15.775955 21420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.782212 21420 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:46257
I20260927 11:08:15.782246 21548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:46257 every 8 connection(s)
I20260927 11:08:15.782608 21420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:15.788604 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21420
I20260927 11:08:15.788717 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:15.788929 17791 external_mini_cluster.cc:1428] Setting key cb5452ca0e4304f54837732f7f14f231
I20260927 11:08:15.792312 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:15.783329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45071 (local address 127.17.95.254:39459)
0927 11:08:15.783401 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:15.783404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:15.783545 (+ 141us) server_negotiation.cc:407] Connection header received
0927 11:08:15.784503 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:15.784505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:15.784515 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:15.784580 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:15.785147 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.785613 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.786622 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.786815 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.786865 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:15.789073 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:15.789074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:15.789074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:15.789097 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:15.790802 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:15.791336 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:15.791337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:15.791337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:15.791396 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:15.791665 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:15.791666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:15.791666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:15.791760 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:15.791811 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:15.792153 (+ 342us) server_negotiation.cc:299] Negotiation successful
0927 11:08:15.792225 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 11:08:15.792387 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:15.783276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39459 (local address 127.17.95.193:45071)
0927 11:08:15.783421 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:15.783462 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:15.784407 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:15.784641 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:15.784645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:15.784887 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:15.785077 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:15.785081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.785705 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.785706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:15.786553 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:15.786555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.786884 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.786884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:15.786978 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:15.787545 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:15.787552 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:15.788985 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:15.790883 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:15.790889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:15.790915 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:15.791268 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:15.791465 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:15.791465 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:15.791466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:15.791598 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:15.791819 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:15.791824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:15.791995 (+ 171us) client_negotiation.cc:769] Sending connection context
0927 11:08:15.792111 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 11:08:15.792221 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260927 11:08:15.792798 21549 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39459
I20260927 11:08:15.792888 21549 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:15.793081 21549 heartbeater.cc:511] Master 127.17.95.254:39459 requested a full tablet report, sending...
I20260927 11:08:15.793603 21358 ts_manager.cc:194] Registered new tserver with Master: 1c6e31d20b844c45b204e367f9c8f7cb (127.17.95.193:46257)
I20260927 11:08:15.794201 21358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:45071
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:15.814074 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39459
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:15.893335 21554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:15.893532 21554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:15.893553 21554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:15.894474 21554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:15.894511 21554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:15.894536 21554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:15.895851 21554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:15.896040 21554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:15.896221 21554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:15.901005 21559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:15.900995 21562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:15.900995 21560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:15.901580 21554 server_base.cc:1034] running on GCE node
I20260927 11:08:15.901793 21554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:15.901988 21554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:15.903119 21554 hybrid_clock.cc:648] HybridClock initialized: now 1790507295903094 us; error 37 us; skew 500 ppm
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:15.905498 21554 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:15.905977 21554 webserver.cc:460] Webserver started at http://127.17.95.194:37339/ using document root <none> and password file <none>
I20260927 11:08:15.906204 21554 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:15.906270 21554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:15.906431 21554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:15.907572 21554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "77a24730bbc2499b908249e70e7987b6"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "48663603c3242a4cd4049468115bfd82"
server_key_iv: "c144dad15c716db7e6606454e4bbc165"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.907934 21554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "77a24730bbc2499b908249e70e7987b6"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "48663603c3242a4cd4049468115bfd82"
server_key_iv: "c144dad15c716db7e6606454e4bbc165"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.909236 21554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:15.910090 21569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.910264 21554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:15.910360 21554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "77a24730bbc2499b908249e70e7987b6"
format_stamp: "Formatted at 2026-09-27 11:08:15 on dist-test-slave-6j0v"
server_key: "48663603c3242a4cd4049468115bfd82"
server_key_iv: "c144dad15c716db7e6606454e4bbc165"
server_key_version: "encryptionkey@0"
I20260927 11:08:15.910470 21554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:15.930747 21554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:15.933353 21554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:15.933568 21554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:15.933827 21554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:15.934227 21554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:15.934276 21554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.934312 21554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:15.934341 21554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:15.940817 21554 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:39137
I20260927 11:08:15.940871 21682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:39137 every 8 connection(s)
I20260927 11:08:15.941183 21554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:08:15 dist-test-slave-6j0v krb5kdc[21313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507295, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:15.948735 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21554
I20260927 11:08:15.948833 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:15.949019 17791 external_mini_cluster.cc:1428] Setting key 624c1c29e90e0066fe2ebe423b71d7a8
I20260927 11:08:15.950122 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:15.941899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:49521 (local address 127.17.95.254:39459)
0927 11:08:15.941960 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:15.941962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:15.942131 (+ 169us) server_negotiation.cc:407] Connection header received
0927 11:08:15.943006 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:15.943007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:15.943016 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:15.943130 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:15.943690 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.944167 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.945096 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.945295 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.945348 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:15.947448 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:15.947449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:15.947450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:15.947456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:15.948676 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:15.949111 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:15.949112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:15.949113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:15.949197 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:15.949489 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:15.949490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:15.949491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:15.949618 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:15.949680 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:15.950002 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 11:08:15.950063 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:08:15.950412 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:15.941861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39459 (local address 127.17.95.194:49521)
0927 11:08:15.942012 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:15.942051 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:15.942915 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:15.943188 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:15.943191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:15.943424 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:15.943614 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:15.943618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.944242 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.944243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:15.945025 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:15.945027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:15.945366 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:15.945367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:15.945494 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:15.945988 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:15.945996 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:15.947370 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:15.948749 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:15.948755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:15.948757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:15.949024 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:15.949281 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:15.949282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:15.949282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:15.949411 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:15.949689 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:15.949695 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:15.949860 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 11:08:15.949980 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 11:08:15.950143 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260927 11:08:15.950860 21683 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39459
I20260927 11:08:15.950989 21683 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:15.951182 21683 heartbeater.cc:511] Master 127.17.95.254:39459 requested a full tablet report, sending...
I20260927 11:08:15.951560 21358 ts_manager.cc:194] Registered new tserver with Master: 77a24730bbc2499b908249e70e7987b6 (127.17.95.194:39137)
I20260927 11:08:15.951907 21358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:49521
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:15.971541 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39459
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:16.051604 21688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:16.051791 21688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:16.051813 21688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:16.052956 21688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:16.052996 21688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:16.053025 21688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:16.054494 21688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:16.054675 21688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:16.054863 21688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:08:16.059790 21688 server_base.cc:1034] running on GCE node
W20260927 11:08:16.059727 21696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:16.059732 21693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:16.059762 21694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:16.060264 21688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:16.060518 21688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:16.061678 21688 hybrid_clock.cc:648] HybridClock initialized: now 1790507296061674 us; error 34 us; skew 500 ppm
Sep 27 11:08:16 dist-test-slave-6j0v krb5kdc[21313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507296, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:16.064095 21688 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:16.064575 21688 webserver.cc:460] Webserver started at http://127.17.95.195:33129/ using document root <none> and password file <none>
I20260927 11:08:16.064811 21688 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:16.064878 21688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:16.065037 21688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:16.066033 21688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "b7ef39cf433a42a1acb16c88689a63f8"
format_stamp: "Formatted at 2026-09-27 11:08:16 on dist-test-slave-6j0v"
server_key: "0666e5f704fd4a2733de9d672902d44f"
server_key_iv: "50addae59f3d9acb390283868fc660d6"
server_key_version: "encryptionkey@0"
I20260927 11:08:16.066394 21688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "b7ef39cf433a42a1acb16c88689a63f8"
format_stamp: "Formatted at 2026-09-27 11:08:16 on dist-test-slave-6j0v"
server_key: "0666e5f704fd4a2733de9d672902d44f"
server_key_iv: "50addae59f3d9acb390283868fc660d6"
server_key_version: "encryptionkey@0"
I20260927 11:08:16.067798 21688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:16.068578 21703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:16.068792 21688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:16.068902 21688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "b7ef39cf433a42a1acb16c88689a63f8"
format_stamp: "Formatted at 2026-09-27 11:08:16 on dist-test-slave-6j0v"
server_key: "0666e5f704fd4a2733de9d672902d44f"
server_key_iv: "50addae59f3d9acb390283868fc660d6"
server_key_version: "encryptionkey@0"
I20260927 11:08:16.068970 21688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:16.087302 21688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:16.089169 21688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:16.089318 21688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:16.089526 21688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:16.089797 21688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:16.089848 21688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:16.089888 21688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:16.089917 21688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:16.095625 21688 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:43713
I20260927 11:08:16.095734 21816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:43713 every 8 connection(s)
I20260927 11:08:16.096006 21688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
Sep 27 11:08:16 dist-test-slave-6j0v krb5kdc[21313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507296, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:16.105132 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:16.096776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:39471 (local address 127.17.95.254:39459)
0927 11:08:16.096855 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:16.096858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:16.097162 (+ 304us) server_negotiation.cc:407] Connection header received
0927 11:08:16.098223 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:16.098225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:16.098234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:16.098289 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:16.098877 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.099438 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.100400 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.100597 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.100650 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:16.102685 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:16.102686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:16.102686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:16.102692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:16.104000 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:16.104354 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:16.104355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:16.104356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:16.104411 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:16.104632 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:16.104633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:16.104633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:16.104729 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:16.104779 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:16.105001 (+ 222us) server_negotiation.cc:299] Negotiation successful
0927 11:08:16.105071 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 11:08:16.105259 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:16.096752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39459 (local address 127.17.95.195:39471)
0927 11:08:16.096998 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:16.097070 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:16.098120 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:16.098366 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:16.098369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:16.098626 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:16.098803 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:16.098807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.099533 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.099533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:16.100327 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:16.100328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.100667 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.100667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:16.100759 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:16.101248 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:16.101256 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:16.102602 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:16.104068 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:16.104073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:16.104075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:16.104276 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:16.104490 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:16.104490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:16.104491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:16.104579 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:16.104783 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:16.104786 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:16.104886 (+ 100us) client_negotiation.cc:769] Sending connection context
0927 11:08:16.104964 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 11:08:16.105072 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
I20260927 11:08:16.105749 21817 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39459
I20260927 11:08:16.105801 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21688
I20260927 11:08:16.105852 21817 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:16.105898 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:16.106065 21817 heartbeater.cc:511] Master 127.17.95.254:39459 requested a full tablet report, sending...
I20260927 11:08:16.106103 17791 external_mini_cluster.cc:1428] Setting key 2c4ccfdd2ed7600d19f4b74d0328fe65
I20260927 11:08:16.106494 21357 ts_manager.cc:194] Registered new tserver with Master: b7ef39cf433a42a1acb16c88689a63f8 (127.17.95.195:43713)
I20260927 11:08:16.106844 21357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:39471
I20260927 11:08:16.107554 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 11:08:16 dist-test-slave-6j0v krb5kdc[21313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 11:08:16 dist-test-slave-6j0v krb5kdc[21313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:16.129905 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:16.122227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.17.95.193:46257)
0927 11:08:16.122381 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:16.122385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:16.122394 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:16.122417 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:16.122419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:16.122431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:16.122496 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:16.123009 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.123837 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.124555 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.124859 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.124915 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:16.126969 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:16.126975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:16.126975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:16.126981 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:16.128643 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:16.129023 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:16.129025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:16.129026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:16.129083 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:16.129313 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:16.129314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:16.129314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:16.129449 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:16.129691 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:16.129745 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 11:08:16.129820 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
Sep 27 11:08:16 dist-test-slave-6j0v krb5kdc[21313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:16.138127 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:16.131105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.17.95.254:39459)
0927 11:08:16.131146 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:16.131148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:16.131182 (+ 34us) server_negotiation.cc:407] Connection header received
0927 11:08:16.131255 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:16.131256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:16.131265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:16.131320 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:16.131788 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.132253 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.132999 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:16.133198 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:16.133280 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:16.135423 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:16.135424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:16.135424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:16.135432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:16.136763 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:16.137147 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:16.137148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:16.137149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:16.137209 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:16.137403 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:16.137404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:16.137404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:16.137614 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:16.137800 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:16.137923 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 11:08:16.138005 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 11:08:16.138360 21358 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32878
I20260927 11:08:16.138662 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21420
I20260927 11:08:16.143672 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21554
I20260927 11:08:16.148524 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21688
I20260927 11:08:16.152647 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21326
2026-09-27T11:08:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1838 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:17 dist-test-slave-6j0v krb5kdc[21829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:17 dist-test-slave-6j0v krb5kdc[21829](info): set up 2 sockets
Sep 27 11:08:17 dist-test-slave-6j0v krb5kdc[21829](info): commencing operation
krb5kdc: starting...
W20260927 11:08:18.873875 17791 mini_kdc.cc:121] Time spent starting KDC: real 2.702s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260927 11:08:18.998030 17791 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.124s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260927 11:08:19.154564 17791 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.156s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260927 11:08:19.393373 17791 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.239s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/test-user.keytab.
W20260927 11:08:20.055740 17791 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.662s user 0.000s sys 0.001s
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
W20260927 11:08:20.342885 17791 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.95.254: real 0.266s user 0.000s sys 0.003s
I20260927 11:08:20.343319 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:38323
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:20.421232 21842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:20.421447 21842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:20.421468 21842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:20.422657 21842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:20.422709 21842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:20.422724 21842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:20.422736 21842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:20.422793 21842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:20.424242 21842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:20.424482 21842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:20.424736 21842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:20.429446 21848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.429463 21850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.429484 21847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:20.429479 21842 server_base.cc:1034] running on GCE node
I20260927 11:08:20.429953 21842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:20.430227 21842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:20.431389 21842 hybrid_clock.cc:648] HybridClock initialized: now 1790507300431378 us; error 28 us; skew 500 ppm
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:20.439306 21842 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:20.439704 21842 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:20.439771 21842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:20.439898 21842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:20.440892 21842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "66a5dea8a9764c3a8fe0ea9052184836"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "58f865c51c7e012e189d8e54b1bd78ad"
server_key_iv: "883195188133de0f3a0b982a98fb810b"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.441239 21842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "66a5dea8a9764c3a8fe0ea9052184836"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "58f865c51c7e012e189d8e54b1bd78ad"
server_key_iv: "883195188133de0f3a0b982a98fb810b"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.442533 21842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:20.443236 21856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.443378 21842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.443487 21842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "66a5dea8a9764c3a8fe0ea9052184836"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "58f865c51c7e012e189d8e54b1bd78ad"
server_key_iv: "883195188133de0f3a0b982a98fb810b"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.443554 21842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:20.449838 21842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:20.451978 21842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:20.452131 21842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:20.456152 21842 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:38323
I20260927 11:08:20.456183 21908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:38323 every 8 connection(s)
I20260927 11:08:20.456439 21842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:20.457275 21909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:20.457793 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21842
I20260927 11:08:20.457926 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:20.458107 17791 external_mini_cluster.cc:1428] Setting key 72d24fef36542b0432b7a47e9b975287
I20260927 11:08:20.460080 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836: Bootstrap starting.
I20260927 11:08:20.460639 21909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:20.460901 21909 log.cc:824] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:20.461540 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836: No bootstrap required, opened a new log
I20260927 11:08:20.463266 21909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a5dea8a9764c3a8fe0ea9052184836" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38323 } }
I20260927 11:08:20.463406 21909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:20.463459 21909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a5dea8a9764c3a8fe0ea9052184836, State: Initialized, Role: FOLLOWER
I20260927 11:08:20.463596 21909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a5dea8a9764c3a8fe0ea9052184836" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38323 } }
I20260927 11:08:20.463667 21909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:20.463709 21909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:20.463754 21909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:20.464257 21909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a5dea8a9764c3a8fe0ea9052184836" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38323 } }
I20260927 11:08:20.464372 21909 leader_election.cc:302] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66a5dea8a9764c3a8fe0ea9052184836; no voters:
I20260927 11:08:20.464535 21909 leader_election.cc:288] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:20.464579 21914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:20.464776 21909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:20.464768 21914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [term 1 LEADER]: Becoming Leader. State: Replica: 66a5dea8a9764c3a8fe0ea9052184836, State: Running, Role: LEADER
I20260927 11:08:20.465232 21914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a5dea8a9764c3a8fe0ea9052184836" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38323 } }
I20260927 11:08:20.465997 21915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66a5dea8a9764c3a8fe0ea9052184836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a5dea8a9764c3a8fe0ea9052184836" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38323 } } }
I20260927 11:08:20.466089 21915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:20.466361 21916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a5dea8a9764c3a8fe0ea9052184836. Latest consensus state: current_term: 1 leader_uuid: "66a5dea8a9764c3a8fe0ea9052184836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a5dea8a9764c3a8fe0ea9052184836" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38323 } } }
I20260927 11:08:20.466665 21916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:20.466519 21925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:20.467137 21925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:20.468786 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.458593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56742 (local address 127.17.95.254:38323)
0927 11:08:20.458737 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.458743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.458767 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:08:20.459206 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.459212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.459257 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.459361 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:20.460121 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.461067 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.461888 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.462289 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.462399 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.464735 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:20.464739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:20.464739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:20.464749 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:20.467226 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.467615 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.467617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.467618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.467709 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.467914 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.467915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.467916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.468169 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:20.468309 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.468469 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.468627 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260927 11:08:20.468912 21925 catalog_manager.cc:1347] Generated new cluster ID: 80af8f8a4b3b4a1d99471fa1b73a2a79
I20260927 11:08:20.468998 21925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:20.475822 21925 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:20.476310 21925 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:20.479846 21925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836: Generated new TSK 0
I20260927 11:08:20.480010 21925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:20.507329 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:38323
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:20.597831 21935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:20.598019 21935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:20.598040 21935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:20.599170 21935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:20.599205 21935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:20.599231 21935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:20.600641 21935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:20.600853 21935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:20.601083 21935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:20.606731 21943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.606735 21941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.606735 21940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:20.607092 21935 server_base.cc:1034] running on GCE node
I20260927 11:08:20.607366 21935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:20.607604 21935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:20.608757 21935 hybrid_clock.cc:648] HybridClock initialized: now 1790507300608745 us; error 35 us; skew 500 ppm
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:20.611363 21935 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:20.611686 21935 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:20.611739 21935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:20.611856 21935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:20.613113 21935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "fdd170a25127428984088a2fdaa8f0dd"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "a2a77ae7ea69f06e4012c2fad431fe97"
server_key_iv: "8a37ee8148328de7dd44ffcb6ce43297"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.613509 21935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "fdd170a25127428984088a2fdaa8f0dd"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "a2a77ae7ea69f06e4012c2fad431fe97"
server_key_iv: "8a37ee8148328de7dd44ffcb6ce43297"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.614976 21935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:20.615814 21949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.615996 21935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:20.616079 21935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "fdd170a25127428984088a2fdaa8f0dd"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "a2a77ae7ea69f06e4012c2fad431fe97"
server_key_iv: "8a37ee8148328de7dd44ffcb6ce43297"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.616142 21935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:20.624563 21935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:20.626442 21935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:20.626544 21935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:20.626747 21935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:20.626997 21935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:20.627028 21935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.627048 21935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:20.627060 21935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.632761 21935 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41409
I20260927 11:08:20.632804 22062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41409 every 8 connection(s)
I20260927 11:08:20.633051 21935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:20.641816 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.633711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:42343 (local address 127.17.95.254:38323)
0927 11:08:20.633769 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.633771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.633972 (+ 201us) server_negotiation.cc:407] Connection header received
0927 11:08:20.634496 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.634498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.634510 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.634555 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:20.635521 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.636027 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.637010 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.637228 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.637309 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.639330 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:20.639332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:20.639332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:20.639340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:20.640521 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.640889 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.640890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.640891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.640951 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.641181 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.641182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.641182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.641307 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:20.641379 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.641640 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.641729 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:08:20.641978 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.633685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:38323 (local address 127.17.95.193:42343)
0927 11:08:20.633852 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:20.633885 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:20.634362 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:20.634633 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:20.634637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:20.634883 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:20.635417 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.635423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.636122 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.636124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.636901 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.636903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.637319 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.637320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.637437 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.637913 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:20.637921 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:20.639244 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:20.640600 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:20.640607 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:20.640609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:20.640819 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:20.641014 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:20.641015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:20.641015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:20.641119 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:20.641370 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:20.641376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:20.641511 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 11:08:20.641608 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 11:08:20.641753 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260927 11:08:20.642236 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 21935
I20260927 11:08:20.642351 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:20.642580 17791 external_mini_cluster.cc:1428] Setting key 888d50cdc043da446a38e8d0fe1bd4bd
I20260927 11:08:20.642644 22063 heartbeater.cc:347] Connected to a master server at 127.17.95.254:38323
I20260927 11:08:20.642737 22063 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:20.642872 22063 heartbeater.cc:511] Master 127.17.95.254:38323 requested a full tablet report, sending...
I20260927 11:08:20.643402 21873 ts_manager.cc:194] Registered new tserver with Master: fdd170a25127428984088a2fdaa8f0dd (127.17.95.193:41409)
I20260927 11:08:20.643981 21873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:42343
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:20.665917 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:38323
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:20.742921 22068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:20.743142 22068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:20.743167 22068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:20.744239 22068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:20.744277 22068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:20.744303 22068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:20.745677 22068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:20.745903 22068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:20.746104 22068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:20.750604 22076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.750604 22074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.750667 22073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:20.750880 22068 server_base.cc:1034] running on GCE node
I20260927 11:08:20.751036 22068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:20.751269 22068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:20.752439 22068 hybrid_clock.cc:648] HybridClock initialized: now 1790507300752422 us; error 31 us; skew 500 ppm
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:20.754817 22068 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:20.755185 22068 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:20.755249 22068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:20.755378 22068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:20.756326 22068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "83a74a086ca54069a1abab098c542418"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "cbd2afb3f4ed7f3ce15f478346573793"
server_key_iv: "3d9b15d4f21d61a8ae583719f7ae76a7"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.756690 22068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "83a74a086ca54069a1abab098c542418"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "cbd2afb3f4ed7f3ce15f478346573793"
server_key_iv: "3d9b15d4f21d61a8ae583719f7ae76a7"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.757869 22068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:20.758495 22082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.758641 22068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:20.758726 22068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "83a74a086ca54069a1abab098c542418"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "cbd2afb3f4ed7f3ce15f478346573793"
server_key_iv: "3d9b15d4f21d61a8ae583719f7ae76a7"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.758795 22068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:20.777630 22068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:20.779680 22068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:20.779829 22068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:20.780067 22068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:20.780370 22068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:20.780418 22068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.780453 22068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:20.780480 22068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.785857 22068 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:44395
I20260927 11:08:20.785933 22195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:44395 every 8 connection(s)
I20260927 11:08:20.786167 22068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:20.789995 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22068
I20260927 11:08:20.790105 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:20.790293 17791 external_mini_cluster.cc:1428] Setting key e1f88599dec75516cb756da96c7d1db9
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:20.795874 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.786799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:40679 (local address 127.17.95.254:38323)
0927 11:08:20.786863 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.786865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.787098 (+ 233us) server_negotiation.cc:407] Connection header received
0927 11:08:20.787660 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.787662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.787676 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.787733 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:20.788648 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.789183 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.790234 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.790535 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.790597 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.793054 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:20.793056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:20.793056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:20.793062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:20.794285 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.794769 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.794770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.794771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.794833 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.795130 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.795131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.795132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.795249 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:20.795309 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.795649 (+ 340us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.795758 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:08:20.796070 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.786797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:38323 (local address 127.17.95.194:40679)
0927 11:08:20.786963 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:20.786996 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:20.787552 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:20.787799 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:20.787802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:20.788050 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:20.788543 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.788548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.789297 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.789298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.790150 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.790153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.790616 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.790617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.790738 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.791268 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:20.791277 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:20.792959 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:20.794361 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:20.794367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:20.794369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:20.794677 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:20.794903 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:20.794903 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:20.794904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:20.795046 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:20.795317 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:20.795323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:20.795486 (+ 163us) client_negotiation.cc:769] Sending connection context
0927 11:08:20.795595 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 11:08:20.795789 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260927 11:08:20.796911 22196 heartbeater.cc:347] Connected to a master server at 127.17.95.254:38323
I20260927 11:08:20.796993 22196 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:20.797147 22196 heartbeater.cc:511] Master 127.17.95.254:38323 requested a full tablet report, sending...
I20260927 11:08:20.797542 21873 ts_manager.cc:194] Registered new tserver with Master: 83a74a086ca54069a1abab098c542418 (127.17.95.194:44395)
I20260927 11:08:20.797878 21873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:40679
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:20.814426 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:38323
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:20.895211 22201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:20.895399 22201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:20.895432 22201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:20.896546 22201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:20.896602 22201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:20.896646 22201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:20.897995 22201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:20.898224 22201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:20.898413 22201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:20.902915 22209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:20.902976 22207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:20.903012 22201 server_base.cc:1034] running on GCE node
W20260927 11:08:20.902932 22206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:20.903360 22201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:20.903569 22201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:20.904729 22201 hybrid_clock.cc:648] HybridClock initialized: now 1790507300904707 us; error 33 us; skew 500 ppm
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:20.907269 22201 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:20.907639 22201 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:20.907709 22201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:20.907866 22201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:20.908958 22201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "7f74f52fb7234c87a54922d7b048db2a"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "aa2059c10d08bba4573d7c6aef93b52b"
server_key_iv: "7dd04e06cb33c07ad773e2dd174482d5"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.909340 22201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "7f74f52fb7234c87a54922d7b048db2a"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "aa2059c10d08bba4573d7c6aef93b52b"
server_key_iv: "7dd04e06cb33c07ad773e2dd174482d5"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.910586 22201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:20.911396 22215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.911564 22201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:20.911659 22201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "7f74f52fb7234c87a54922d7b048db2a"
format_stamp: "Formatted at 2026-09-27 11:08:20 on dist-test-slave-6j0v"
server_key: "aa2059c10d08bba4573d7c6aef93b52b"
server_key_iv: "7dd04e06cb33c07ad773e2dd174482d5"
server_key_version: "encryptionkey@0"
I20260927 11:08:20.911734 22201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:20.918418 22201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:20.920331 22201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:20.920483 22201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:20.920688 22201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:20.920981 22201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:20.921036 22201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.921074 22201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:20.921104 22201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:20.927062 22201 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:32791
I20260927 11:08:20.927125 22328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:32791 every 8 connection(s)
I20260927 11:08:20.927383 22201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:20.928439 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22201
I20260927 11:08:20.928560 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:20.928742 17791 external_mini_cluster.cc:1428] Setting key 800a73eb2722918e7d175640c5b99f01
Sep 27 11:08:20 dist-test-slave-6j0v krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507300, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:20.936940 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.928075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:55639 (local address 127.17.95.254:38323)
0927 11:08:20.928136 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.928139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.928369 (+ 230us) server_negotiation.cc:407] Connection header received
0927 11:08:20.928868 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.928870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.928883 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.928928 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:20.930042 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.930586 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.931776 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.931986 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.932073 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.934227 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:20.934228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:20.934228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:20.934235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:20.935625 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.936026 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.936026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.936027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.936073 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.936326 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.936327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.936327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.936448 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:20.936524 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.936802 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.936871 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:20.937119 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.928078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:38323 (local address 127.17.95.195:55639)
0927 11:08:20.928233 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:20.928267 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:20.928766 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:20.929012 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:20.929017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:20.929341 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:20.929929 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.929934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.930678 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.930679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.931682 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.931684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.932093 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.932094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.932211 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.932733 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:20.932741 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:20.934142 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:20.935706 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:20.935711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:20.935731 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:20.935953 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:20.936125 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:20.936126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:20.936126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:20.936224 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:20.936544 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:20.936550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:20.936666 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 11:08:20.936768 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 11:08:20.936903 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260927 11:08:20.937683 22329 heartbeater.cc:347] Connected to a master server at 127.17.95.254:38323
I20260927 11:08:20.937790 22329 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:20.937970 22329 heartbeater.cc:511] Master 127.17.95.254:38323 requested a full tablet report, sending...
I20260927 11:08:20.938372 21873 ts_manager.cc:194] Registered new tserver with Master: 7f74f52fb7234c87a54922d7b048db2a (127.17.95.195:32791)
I20260927 11:08:20.938771 21873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:55639
I20260927 11:08:20.940424 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:08:20.947465 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.941656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.17.95.254:38323)
0927 11:08:20.941717 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.941719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.941816 (+ 97us) server_negotiation.cc:407] Connection header received
0927 11:08:20.941866 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.941867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.941875 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.941918 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:20.942486 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.942920 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.943724 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.943897 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.943949 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.945183 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:20.945184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:20.945185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:20.945190 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:20.946396 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.946703 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.946704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.946704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.946750 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:20.946977 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:20.946978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:20.946978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:20.947155 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:20.947235 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.947355 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.947408 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:08:20.949076 21873 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:20.950572 21873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:20.957057 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.952842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.17.95.195:32791)
0927 11:08:20.953077 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.953081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.953592 (+ 511us) server_negotiation.cc:407] Connection header received
0927 11:08:20.953640 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.953642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.953656 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.953734 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:20.953872 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.954880 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.956010 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.956701 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.956773 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.956827 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.956932 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.956977 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":92,"threads_started":1}
I20260927 11:08:20.957067 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.952651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:32791 (local address 127.0.0.1:41936)
0927 11:08:20.953538 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:20.953546 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:20.953589 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:20.953740 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:20.953741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:20.953743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:20.953794 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.953795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.954965 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.954967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.955921 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.955924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.956793 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.956793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.956846 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.956856 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 11:08:20.956940 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 11:08:20.956986 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":8,"thread_start_us":79,"threads_started":1}
I20260927 11:08:20.957185 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.952651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41409 (local address 127.0.0.1:36298)
0927 11:08:20.953395 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:20.953404 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:20.953457 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:20.953738 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:20.953740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:20.953742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:20.953784 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.953786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.954626 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.954627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.956073 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.956076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.956960 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.956960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.957008 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.957011 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:20.957074 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 11:08:20.957113 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":135,"thread_start_us":56,"threads_started":1}
I20260927 11:08:20.957475 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.952891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.17.95.193:41409)
0927 11:08:20.953080 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.953083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.953500 (+ 417us) server_negotiation.cc:407] Connection header received
0927 11:08:20.953560 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.953563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.953578 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.953653 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:20.953842 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.954532 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.956158 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.956871 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.956949 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.956998 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.957345 (+ 347us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.957379 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260927 11:08:20.958052 22263 tablet_service.cc:1467] Processing CreateTablet for tablet a10585aa2ec144ac8ff21d827994522e (DEFAULT_TABLE table=test-table [id=8f7f2782ce4540888d09e78638388ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:20.958194 22337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.952600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:44395 (local address 127.0.0.1:36530)
0927 11:08:20.952776 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:20.952815 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:20.952882 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:20.953298 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:20.953301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:20.953304 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:20.953366 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.953369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.954704 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.954704 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.956075 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:20.956076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.957962 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.957963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:20.958023 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.958026 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:20.958082 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:08:20.958121 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":119,"thread_start_us":75,"threads_started":1}
I20260927 11:08:20.958231 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:20.952719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.17.95.194:44395)
0927 11:08:20.953146 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:20.953149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:20.953157 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:20.953188 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:20.953189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:20.953201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:20.953293 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:20.953455 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.954611 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.957033 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:20.957801 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:20.957880 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:20.957930 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:20.958110 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 11:08:20.958151 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":65,"threads_started":1}
I20260927 11:08:20.958201 21997 tablet_service.cc:1467] Processing CreateTablet for tablet a10585aa2ec144ac8ff21d827994522e (DEFAULT_TABLE table=test-table [id=8f7f2782ce4540888d09e78638388ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:20.958405 22263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a10585aa2ec144ac8ff21d827994522e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:20.958499 21997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a10585aa2ec144ac8ff21d827994522e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:20.960815 22343 tablet_bootstrap.cc:492] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Bootstrap starting.
I20260927 11:08:20.961380 22344 tablet_bootstrap.cc:492] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: Bootstrap starting.
I20260927 11:08:20.961508 22343 tablet_bootstrap.cc:654] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:20.961532 22130 tablet_service.cc:1467] Processing CreateTablet for tablet a10585aa2ec144ac8ff21d827994522e (DEFAULT_TABLE table=test-table [id=8f7f2782ce4540888d09e78638388ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:20.961850 22130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a10585aa2ec144ac8ff21d827994522e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:20.962131 22344 tablet_bootstrap.cc:654] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:20.962137 22343 log.cc:824] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:20.962723 22344 log.cc:824] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:20.962971 22343 tablet_bootstrap.cc:492] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: No bootstrap required, opened a new log
I20260927 11:08:20.963038 22343 ts_tablet_manager.cc:1397] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:20.963500 22344 tablet_bootstrap.cc:492] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: No bootstrap required, opened a new log
I20260927 11:08:20.963578 22344 ts_tablet_manager.cc:1397] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:20.965035 22343 raft_consensus.cc:348] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:20.965196 22343 raft_consensus.cc:374] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:20.965232 22343 raft_consensus.cc:729] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f74f52fb7234c87a54922d7b048db2a, State: Initialized, Role: FOLLOWER
I20260927 11:08:20.965343 22343 consensus_queue.cc:260] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:20.965626 22329 heartbeater.cc:503] Master 127.17.95.254:38323 was elected leader, sending a full tablet report...
I20260927 11:08:20.965615 22343 ts_tablet_manager.cc:1428] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:20.965684 22344 raft_consensus.cc:348] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:20.965910 22347 tablet_bootstrap.cc:492] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Bootstrap starting.
I20260927 11:08:20.965936 22344 raft_consensus.cc:374] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:20.965991 22344 raft_consensus.cc:729] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdd170a25127428984088a2fdaa8f0dd, State: Initialized, Role: FOLLOWER
I20260927 11:08:20.966452 22344 consensus_queue.cc:260] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:20.966523 22347 tablet_bootstrap.cc:654] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:20.966893 22344 ts_tablet_manager.cc:1428] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:08:20.967120 22063 heartbeater.cc:503] Master 127.17.95.254:38323 was elected leader, sending a full tablet report...
I20260927 11:08:20.967146 22347 log.cc:824] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:20.967991 22347 tablet_bootstrap.cc:492] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: No bootstrap required, opened a new log
I20260927 11:08:20.968058 22347 ts_tablet_manager.cc:1397] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:08:20.969340 22347 raft_consensus.cc:348] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:20.969466 22347 raft_consensus.cc:374] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:20.969520 22347 raft_consensus.cc:729] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a74a086ca54069a1abab098c542418, State: Initialized, Role: FOLLOWER
I20260927 11:08:20.969619 22347 consensus_queue.cc:260] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:20.969852 22347 ts_tablet_manager.cc:1428] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:08:20.969874 22196 heartbeater.cc:503] Master 127.17.95.254:38323 was elected leader, sending a full tablet report...
W20260927 11:08:21.036669 22197 tablet.cc:2369] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:21.083011 22349 raft_consensus.cc:482] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:21.083189 22349 raft_consensus.cc:504] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:21.083599 22349 leader_election.cc:288] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f74f52fb7234c87a54922d7b048db2a (127.17.95.195:32791), 83a74a086ca54069a1abab098c542418 (127.17.95.194:44395)
I20260927 11:08:21.086454 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.083789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:44395 (local address 127.17.95.193:51575)
0927 11:08:21.083950 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.083959 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.084008 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.084224 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.084225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.084228 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:21.084295 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:21.084297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.084893 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.084894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:21.085616 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:21.085617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.086197 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.086198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:21.086269 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:21.086273 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.086325 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.086353 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260927 11:08:21.086459 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.083878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:51575 (local address 127.17.95.194:44395)
0927 11:08:21.083925 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.083927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.084060 (+ 133us) server_negotiation.cc:407] Connection header received
0927 11:08:21.084094 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.084096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.084126 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.084204 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:21.084361 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.084807 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.085702 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.086115 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.086194 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:21.086205 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.086341 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.086387 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 11:08:21.086516 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.083814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:56721 (local address 127.17.95.195:32791)
0927 11:08:21.083866 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.083868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.083876 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:21.083902 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.083903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.083928 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.083996 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:21.084249 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.084755 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.085750 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.086129 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.086213 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:21.086221 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.086409 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.086444 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 11:08:21.086588 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.083726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:32791 (local address 127.17.95.193:56721)
0927 11:08:21.083789 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.083800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.083846 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.084088 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.084089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.084092 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:21.084170 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:21.084173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.084839 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.084840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:21.085629 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:21.085632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.086230 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.086231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:21.086330 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:21.086334 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.086396 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.086431 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":3}
I20260927 11:08:21.087020 22283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10585aa2ec144ac8ff21d827994522e" candidate_uuid: "fdd170a25127428984088a2fdaa8f0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f74f52fb7234c87a54922d7b048db2a" is_pre_election: true
I20260927 11:08:21.087028 22150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10585aa2ec144ac8ff21d827994522e" candidate_uuid: "fdd170a25127428984088a2fdaa8f0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a74a086ca54069a1abab098c542418" is_pre_election: true
I20260927 11:08:21.087177 22283 raft_consensus.cc:2454] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdd170a25127428984088a2fdaa8f0dd in term 0.
I20260927 11:08:21.087221 22150 raft_consensus.cc:2454] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdd170a25127428984088a2fdaa8f0dd in term 0.
I20260927 11:08:21.087419 21952 leader_election.cc:302] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f74f52fb7234c87a54922d7b048db2a, fdd170a25127428984088a2fdaa8f0dd; no voters:
I20260927 11:08:21.087563 22349 raft_consensus.cc:2781] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:21.087600 22349 raft_consensus.cc:482] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:21.087615 22349 raft_consensus.cc:3037] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.088229 22349 raft_consensus.cc:504] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:21.088366 22349 leader_election.cc:288] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [CANDIDATE]: Term 1 election: Requested vote from peers 7f74f52fb7234c87a54922d7b048db2a (127.17.95.195:32791), 83a74a086ca54069a1abab098c542418 (127.17.95.194:44395)
I20260927 11:08:21.088516 22283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10585aa2ec144ac8ff21d827994522e" candidate_uuid: "fdd170a25127428984088a2fdaa8f0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f74f52fb7234c87a54922d7b048db2a"
I20260927 11:08:21.088549 22150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a10585aa2ec144ac8ff21d827994522e" candidate_uuid: "fdd170a25127428984088a2fdaa8f0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a74a086ca54069a1abab098c542418"
I20260927 11:08:21.088599 22283 raft_consensus.cc:3037] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.088603 22150 raft_consensus.cc:3037] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.089304 22150 raft_consensus.cc:2454] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdd170a25127428984088a2fdaa8f0dd in term 1.
I20260927 11:08:21.089314 22283 raft_consensus.cc:2454] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdd170a25127428984088a2fdaa8f0dd in term 1.
I20260927 11:08:21.089464 21953 leader_election.cc:302] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83a74a086ca54069a1abab098c542418, fdd170a25127428984088a2fdaa8f0dd; no voters:
I20260927 11:08:21.089586 22349 raft_consensus.cc:2781] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:21.089782 22349 raft_consensus.cc:686] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 1 LEADER]: Becoming Leader. State: Replica: fdd170a25127428984088a2fdaa8f0dd, State: Running, Role: LEADER
I20260927 11:08:21.089877 22349 consensus_queue.cc:237] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } }
I20260927 11:08:21.090718 21873 catalog_manager.cc:5495] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd reported cstate change: term changed from 0 to 1, leader changed from <none> to fdd170a25127428984088a2fdaa8f0dd (127.17.95.193). New cstate: current_term: 1 leader_uuid: "fdd170a25127428984088a2fdaa8f0dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdd170a25127428984088a2fdaa8f0dd" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41409 } health_report { overall_health: HEALTHY } } }
I20260927 11:08:21.102399 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.100486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.17.95.193:41409)
0927 11:08:21.100524 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.100526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.100546 (+ 20us) server_negotiation.cc:407] Connection header received
0927 11:08:21.100608 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.100608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.100622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.100670 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:21.100774 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.101179 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.101827 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.101999 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.102046 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:21.102158 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:21.102242 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:21.102293 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.102329 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.102342 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:08:21.103912 22283 raft_consensus.cc:1261] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 1 FOLLOWER]: Refusing update from remote peer fdd170a25127428984088a2fdaa8f0dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:21.103928 22150 raft_consensus.cc:1261] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 1 FOLLOWER]: Refusing update from remote peer fdd170a25127428984088a2fdaa8f0dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:21.104247 22353 consensus_queue.cc:1035] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "83a74a086ca54069a1abab098c542418" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:21.104383 22349 consensus_queue.cc:1035] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f74f52fb7234c87a54922d7b048db2a" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:21.106910 22356 mvcc.cc:205] Tried to move back new op lower bound from 7333917905319440384 to 7333917905264709632. Current Snapshot: MvccSnapshot[applied={T|T < 7333917905319440384}]
I20260927 11:08:21.107646 22357 mvcc.cc:205] Tried to move back new op lower bound from 7333917905319440384 to 7333917905264709632. Current Snapshot: MvccSnapshot[applied={T|T < 7333917905319440384}]
I20260927 11:08:21.108911 21872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:21.109001 21872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:21.109688 21872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66a5dea8a9764c3a8fe0ea9052184836: Sending DeleteTablet for 3 replicas of tablet a10585aa2ec144ac8ff21d827994522e
I20260927 11:08:21.110018 22130 tablet_service.cc:1514] Processing DeleteTablet for tablet a10585aa2ec144ac8ff21d827994522e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:21 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:36530
I20260927 11:08:21.110054 22263 tablet_service.cc:1514] Processing DeleteTablet for tablet a10585aa2ec144ac8ff21d827994522e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:21 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:41936
I20260927 11:08:21.110049 21997 tablet_service.cc:1514] Processing DeleteTablet for tablet a10585aa2ec144ac8ff21d827994522e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:21 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:36298
I20260927 11:08:21.110251 22367 tablet_replica.cc:330] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd: stopping tablet replica
I20260927 11:08:21.110351 22367 raft_consensus.cc:2229] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:21.110353 22365 tablet_replica.cc:330] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: stopping tablet replica
I20260927 11:08:21.110443 22367 raft_consensus.cc:2258] T a10585aa2ec144ac8ff21d827994522e P fdd170a25127428984088a2fdaa8f0dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:21.110512 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21935
I20260927 11:08:21.110522 22365 raft_consensus.cc:2229] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:21.111918 22365 raft_consensus.cc:2258] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:21.112382 22365 ts_tablet_manager.cc:1905] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:21.113776 22365 ts_tablet_manager.cc:1918] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:21.113865 22365 log.cc:1196] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/a10585aa2ec144ac8ff21d827994522e
I20260927 11:08:21.114157 22365 ts_tablet_manager.cc:1939] T a10585aa2ec144ac8ff21d827994522e P 83a74a086ca54069a1abab098c542418: Deleting consensus metadata
I20260927 11:08:21.114528 21860 catalog_manager.cc:4841] TS 83a74a086ca54069a1abab098c542418 (127.17.95.194:44395): tablet a10585aa2ec144ac8ff21d827994522e (table test-table [id=8f7f2782ce4540888d09e78638388ebc]) successfully deleted
I20260927 11:08:21.115181 22366 tablet_replica.cc:330] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: stopping tablet replica
I20260927 11:08:21.115279 22366 raft_consensus.cc:2229] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:21.115370 22366 raft_consensus.cc:2258] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:21.115706 22366 ts_tablet_manager.cc:1905] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 11:08:21.116174 21857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:41409 (error 108)
I20260927 11:08:21.116341 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22068
I20260927 11:08:21.116569 22337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.116476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41409 (local address 127.0.0.1:36308)
0927 11:08:21.116516 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.116543 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 11:08:21.116663 21857 catalog_manager.cc:4584] TS fdd170a25127428984088a2fdaa8f0dd (127.17.95.193:41409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a10585aa2ec144ac8ff21d827994522e: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41409: connect: Connection refused (error 111)
I20260927 11:08:21.117702 22366 ts_tablet_manager.cc:1918] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:21.117756 22366 log.cc:1196] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/a10585aa2ec144ac8ff21d827994522e
I20260927 11:08:21.118013 22366 ts_tablet_manager.cc:1939] T a10585aa2ec144ac8ff21d827994522e P 7f74f52fb7234c87a54922d7b048db2a: Deleting consensus metadata
I20260927 11:08:21.118369 21859 catalog_manager.cc:4841] TS 7f74f52fb7234c87a54922d7b048db2a (127.17.95.195:32791): tablet a10585aa2ec144ac8ff21d827994522e (table test-table [id=8f7f2782ce4540888d09e78638388ebc]) successfully deleted
I20260927 11:08:21.122048 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22201
I20260927 11:08:21.126318 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 21842
2026-09-27T11:08:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4973 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T11:08:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:21Z Disabled control of system clock
I20260927 11:08:21.152302 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:42675
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:42675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 11:08:21.228793 22372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:21.228983 22372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:21.229005 22372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:21.230000 22372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:21.230028 22372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:21.230041 22372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:21.230052 22372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:21.230064 22372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:21.231213 22372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:42675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:42675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:21.231390 22372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:21.231586 22372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:21.235728 22380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.235739 22377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.235795 22378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:21.235942 22372 server_base.cc:1034] running on GCE node
I20260927 11:08:21.236166 22372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:21.236460 22372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:21.237609 22372 hybrid_clock.cc:648] HybridClock initialized: now 1790507301237592 us; error 36 us; skew 500 ppm
I20260927 11:08:21.238897 22372 webserver.cc:460] Webserver started at http://127.17.95.254:44507/ using document root <none> and password file <none>
I20260927 11:08:21.239181 22372 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:21.239244 22372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:21.239372 22372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:21.240336 22372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "463e66d315fd444faa3c9aae6ac10cd6"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "3f5a60baad8bd3360ba91ea7573abb60"
server_key_iv: "582a54c1c732474f2348cc122a5341e3"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.240669 22372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "463e66d315fd444faa3c9aae6ac10cd6"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "3f5a60baad8bd3360ba91ea7573abb60"
server_key_iv: "582a54c1c732474f2348cc122a5341e3"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.241832 22372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:21.242465 22386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.242619 22372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:21.242707 22372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "463e66d315fd444faa3c9aae6ac10cd6"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "3f5a60baad8bd3360ba91ea7573abb60"
server_key_iv: "582a54c1c732474f2348cc122a5341e3"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.242780 22372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:21.245986 22372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:21.246220 22372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:21.246346 22372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:21.249984 22372 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:42675
I20260927 11:08:21.250010 22438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:42675 every 8 connection(s)
I20260927 11:08:21.250310 22372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:21.250929 22439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:21.253541 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6: Bootstrap starting.
I20260927 11:08:21.254009 22439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:21.254230 22439 log.cc:824] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:21.254777 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6: No bootstrap required, opened a new log
I20260927 11:08:21.255872 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22372
I20260927 11:08:21.255971 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:21.256153 17791 external_mini_cluster.cc:1428] Setting key 15704a9087a1f91c2183348d7d10914a
I20260927 11:08:21.256554 22439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463e66d315fd444faa3c9aae6ac10cd6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42675 } }
I20260927 11:08:21.256726 22439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:21.256776 22439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463e66d315fd444faa3c9aae6ac10cd6, State: Initialized, Role: FOLLOWER
I20260927 11:08:21.256913 22439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463e66d315fd444faa3c9aae6ac10cd6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42675 } }
I20260927 11:08:21.256984 22439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:21.257022 22439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:21.257071 22439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.257794 22439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463e66d315fd444faa3c9aae6ac10cd6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42675 } }
I20260927 11:08:21.257901 22439 leader_election.cc:302] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 463e66d315fd444faa3c9aae6ac10cd6; no voters:
I20260927 11:08:21.258046 22439 leader_election.cc:288] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:21.258085 22444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:21.258224 22444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [term 1 LEADER]: Becoming Leader. State: Replica: 463e66d315fd444faa3c9aae6ac10cd6, State: Running, Role: LEADER
I20260927 11:08:21.258277 22439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:21.258314 22444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463e66d315fd444faa3c9aae6ac10cd6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42675 } }
I20260927 11:08:21.258664 22444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 463e66d315fd444faa3c9aae6ac10cd6. Latest consensus state: current_term: 1 leader_uuid: "463e66d315fd444faa3c9aae6ac10cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463e66d315fd444faa3c9aae6ac10cd6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42675 } } }
I20260927 11:08:21.258756 22444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:21.258791 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.256613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.17.95.254:42675)
0927 11:08:21.256919 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.256927 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.256955 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:08:21.257083 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.257090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.257132 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.257244 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.257921 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.257924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.257924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.257928 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.257978 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.257992 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.258054 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.258320 (+ 266us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.258376 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
I20260927 11:08:21.258677 22445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "463e66d315fd444faa3c9aae6ac10cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463e66d315fd444faa3c9aae6ac10cd6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42675 } } }
I20260927 11:08:21.259033 22445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:21.259784 22451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:21.260349 22451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:21.261662 22451 catalog_manager.cc:1347] Generated new cluster ID: 99fab81b808d49d993f58c50d44fb60b
I20260927 11:08:21.261708 22451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:21.268579 22451 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:21.268714 22451 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:21.274341 22451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6: Generated new TSK 0
I20260927 11:08:21.274501 22451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:21.279771 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42675
--builtin_ntp_servers=127.17.95.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 11:08:21.360301 22463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:21.360491 22463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:21.360514 22463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:21.361645 22463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:21.361686 22463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:21.361713 22463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:21.362987 22463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:21.363265 22463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:21.363583 22463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:21.369299 22469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.369300 22471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.369396 22468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:21.369601 22463 server_base.cc:1034] running on GCE node
I20260927 11:08:21.369798 22463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:21.370074 22463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:21.371218 22463 hybrid_clock.cc:648] HybridClock initialized: now 1790507301371207 us; error 27 us; skew 500 ppm
I20260927 11:08:21.372509 22463 webserver.cc:460] Webserver started at http://127.17.95.193:42453/ using document root <none> and password file <none>
I20260927 11:08:21.372757 22463 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:21.372800 22463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:21.372929 22463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:21.374037 22463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "323a39c92a8f49a19c0e6c6e2a14eda7"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "440e0edc55bd76750111bf07ad1fe9fa"
server_key_iv: "9a0ce9aed78293173b2d6dc5e653c491"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.374362 22463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "323a39c92a8f49a19c0e6c6e2a14eda7"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "440e0edc55bd76750111bf07ad1fe9fa"
server_key_iv: "9a0ce9aed78293173b2d6dc5e653c491"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.375648 22463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:21.376386 22477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.376564 22463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:21.376643 22463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "323a39c92a8f49a19c0e6c6e2a14eda7"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "440e0edc55bd76750111bf07ad1fe9fa"
server_key_iv: "9a0ce9aed78293173b2d6dc5e653c491"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.376711 22463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:21.379952 22463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:21.380187 22463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:21.380301 22463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:21.380522 22463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:21.380833 22463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:21.380867 22463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.380884 22463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:21.380923 22463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.386344 22463 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:34047
I20260927 11:08:21.386379 22590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:34047 every 8 connection(s)
I20260927 11:08:21.386686 22463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:21.389264 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.387329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:37317 (local address 127.17.95.254:42675)
0927 11:08:21.387374 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.387377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.387611 (+ 234us) server_negotiation.cc:407] Connection header received
0927 11:08:21.388003 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.388004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.388009 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.388059 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.389101 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.389102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.389102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.389105 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.389114 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.389117 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.389154 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.389204 (+ 50us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.389216 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:21.389448 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.387346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42675 (local address 127.17.95.193:37317)
0927 11:08:21.387504 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.387533 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.387931 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.388105 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.388108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.388116 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.388986 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.388986 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.389011 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.389036 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.389154 (+ 118us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.389156 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.389158 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.389205 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.389246 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260927 11:08:21.389895 22591 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42675
I20260927 11:08:21.389961 22591 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:21.390107 22591 heartbeater.cc:511] Master 127.17.95.254:42675 requested a full tablet report, sending...
I20260927 11:08:21.390594 22403 ts_manager.cc:194] Registered new tserver with Master: 323a39c92a8f49a19c0e6c6e2a14eda7 (127.17.95.193:34047)
I20260927 11:08:21.394076 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22463
I20260927 11:08:21.394183 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:21.394358 17791 external_mini_cluster.cc:1428] Setting key 6e2424f67f975c5f2b3b952d8735c3d0
I20260927 11:08:21.395215 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42675
--builtin_ntp_servers=127.17.95.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 11:08:21.472277 22594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:21.472489 22594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:21.472525 22594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:21.473592 22594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:21.473647 22594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:21.473687 22594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:21.474910 22594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:21.475150 22594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:21.475390 22594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:21.480348 22602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.480358 22600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:21.480438 22594 server_base.cc:1034] running on GCE node
W20260927 11:08:21.480388 22599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:21.480801 22594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:21.481050 22594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:21.482213 22594 hybrid_clock.cc:648] HybridClock initialized: now 1790507301482188 us; error 35 us; skew 500 ppm
I20260927 11:08:21.483525 22594 webserver.cc:460] Webserver started at http://127.17.95.194:41749/ using document root <none> and password file <none>
I20260927 11:08:21.483749 22594 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:21.483814 22594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:21.483966 22594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:21.484942 22594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "b17bbc3089da02c71c5ebb2310909fd9"
server_key_iv: "1e58e53a750876edb3d11a17ede96df4"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.485288 22594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "b17bbc3089da02c71c5ebb2310909fd9"
server_key_iv: "1e58e53a750876edb3d11a17ede96df4"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.486584 22594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:08:21.487315 22608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.487471 22594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:21.487560 22594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "b17bbc3089da02c71c5ebb2310909fd9"
server_key_iv: "1e58e53a750876edb3d11a17ede96df4"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.487632 22594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:21.491003 22594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:21.491252 22594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:21.491372 22594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:21.491575 22594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:21.491848 22594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:21.491894 22594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.491931 22594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:21.491966 22594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.497243 22594 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:42273
I20260927 11:08:21.497306 22721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:42273 every 8 connection(s)
I20260927 11:08:21.497615 22594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:21.499475 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22594
I20260927 11:08:21.499605 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:21.499784 17791 external_mini_cluster.cc:1428] Setting key 9b51961aa3f028ed367491093abab5f3
I20260927 11:08:21.500520 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.498304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:52151 (local address 127.17.95.254:42675)
0927 11:08:21.498351 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.498354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.498601 (+ 247us) server_negotiation.cc:407] Connection header received
0927 11:08:21.498998 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.499000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.499006 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.499053 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.500270 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.500271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.500272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.500275 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.500291 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.500295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.500368 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.500434 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.500449 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:21.500677 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.498302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42675 (local address 127.17.95.194:52151)
0927 11:08:21.498490 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.498524 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.498907 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.499156 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.499159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.499169 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.500147 (+ 978us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.500148 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.500173 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.500198 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.500362 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.500368 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.500370 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.500437 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.500487 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260927 11:08:21.500748 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42675
--builtin_ntp_servers=127.17.95.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 11:08:21.501102 22722 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42675
I20260927 11:08:21.501200 22722 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:21.501345 22722 heartbeater.cc:511] Master 127.17.95.254:42675 requested a full tablet report, sending...
I20260927 11:08:21.501667 22403 ts_manager.cc:194] Registered new tserver with Master: aa9c3e8bbbd745108ec082ce6efd7ce6 (127.17.95.194:42273)
W20260927 11:08:21.580052 22725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:21.580243 22725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:21.580266 22725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:21.581384 22725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:21.581426 22725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:21.581461 22725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:21.582669 22725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:21.582834 22725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:21.583050 22725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:21.587828 22730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.587839 22733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.587839 22731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:21.588136 22725 server_base.cc:1034] running on GCE node
I20260927 11:08:21.588307 22725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:21.588498 22725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:21.589686 22725 hybrid_clock.cc:648] HybridClock initialized: now 1790507301589661 us; error 30 us; skew 500 ppm
I20260927 11:08:21.591013 22725 webserver.cc:460] Webserver started at http://127.17.95.195:34609/ using document root <none> and password file <none>
I20260927 11:08:21.591253 22725 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:21.591317 22725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:21.591450 22725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:21.592423 22725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "f4a8b8e1044945eeb001fc55607e8543"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "e34678264778b4b787cdd6ae2cd4bec2"
server_key_iv: "1706bd61e8875e499b08510a5e16a5ce"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.592762 22725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "f4a8b8e1044945eeb001fc55607e8543"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "e34678264778b4b787cdd6ae2cd4bec2"
server_key_iv: "1706bd61e8875e499b08510a5e16a5ce"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.593942 22725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:21.594630 22739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.594800 22725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:21.594885 22725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "f4a8b8e1044945eeb001fc55607e8543"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "e34678264778b4b787cdd6ae2cd4bec2"
server_key_iv: "1706bd61e8875e499b08510a5e16a5ce"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.594950 22725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:21.597971 22725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:21.598201 22725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:21.598320 22725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:21.598519 22725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:21.598801 22725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:21.598848 22725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.598886 22725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:21.598914 22725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.604315 22725 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:41849
I20260927 11:08:21.604380 22852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:41849 every 8 connection(s)
I20260927 11:08:21.604653 22725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:21.605265 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22725
I20260927 11:08:21.605381 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:21.605577 17791 external_mini_cluster.cc:1428] Setting key c96c520c6d529e9dade7fc8406fe94e8
I20260927 11:08:21.607595 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.605290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:36923 (local address 127.17.95.254:42675)
0927 11:08:21.605359 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.605362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.605570 (+ 208us) server_negotiation.cc:407] Connection header received
0927 11:08:21.606153 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.606154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.606160 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.606211 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.607396 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.607397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.607398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.607401 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.607412 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.607416 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.607462 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.607533 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.607545 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:08:21.607836 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.605246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42675 (local address 127.17.95.195:36923)
0927 11:08:21.605448 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.605487 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.606046 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.606279 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.606283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.606293 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.607278 (+ 985us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.607278 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.607304 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.607328 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.607460 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.607462 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.607464 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.607528 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.607580 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260927 11:08:21.608219 22853 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42675
I20260927 11:08:21.608289 22853 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:21.608438 22853 heartbeater.cc:511] Master 127.17.95.254:42675 requested a full tablet report, sending...
I20260927 11:08:21.608754 22403 ts_manager.cc:194] Registered new tserver with Master: f4a8b8e1044945eeb001fc55607e8543 (127.17.95.195:41849)
I20260927 11:08:21.617096 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:08:21.619020 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.618166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.17.95.254:42675)
0927 11:08:21.618201 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.618202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.618293 (+ 91us) server_negotiation.cc:407] Connection header received
0927 11:08:21.618339 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.618339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.618342 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.618386 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.618889 (+ 503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.618889 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.618890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.618892 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.618902 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.618905 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.618940 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.618971 (+ 31us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.618981 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:21.620446 22403 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:21.621903 22403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:21.625187 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.623720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:41849 (local address 127.0.0.1:32998)
0927 11:08:21.623839 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.623876 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.623929 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.624288 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.624293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.624298 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.624845 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.624846 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.624850 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.624859 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.625040 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.625044 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.625047 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.625104 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.625116 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":81,"threads_started":1}
I20260927 11:08:21.625214 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.623821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.17.95.195:41849)
0927 11:08:21.624168 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.624171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.624180 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:21.624201 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.624203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.624210 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.624282 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.624939 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.624942 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.624942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.624945 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.624970 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.624981 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.625040 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.625117 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.625131 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":76,"threads_started":1}
I20260927 11:08:21.625404 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.623972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.17.95.193:34047)
0927 11:08:21.624134 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.624138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.624189 (+ 51us) server_negotiation.cc:407] Connection header received
0927 11:08:21.624249 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.624251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.624259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.624322 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.624904 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.624910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.624910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.624914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.624949 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.624975 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.625041 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.625300 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.625316 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260927 11:08:21.625188 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.623720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:34047 (local address 127.0.0.1:37928)
0927 11:08:21.624136 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.624146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.624193 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.624329 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.624329 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.624331 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.624792 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.624793 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.624806 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.624825 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.625040 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.625044 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.625047 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.625103 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.625117 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":3,"thread_start_us":73,"threads_started":1}
I20260927 11:08:21.625928 22787 tablet_service.cc:1467] Processing CreateTablet for tablet c9e849a5c4c34ec6b4b22ceb7249f504 (DEFAULT_TABLE table=test-table [id=15eb37a3ce8f48f9a227bd0d8525dfe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:21.626257 22787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e849a5c4c34ec6b4b22ceb7249f504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:21.626271 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.623913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:42273 (local address 127.0.0.1:37342)
0927 11:08:21.625090 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.625100 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.625141 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.625534 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.625535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.625541 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.625923 (+ 382us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.625923 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.625930 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.625942 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.626139 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.626140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.626140 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.626196 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.626210 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"thread_start_us":58,"threads_started":1}
I20260927 11:08:21.626292 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.624002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.17.95.194:42273)
0927 11:08:21.624162 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.624166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.625190 (+ 1024us) server_negotiation.cc:407] Connection header received
0927 11:08:21.625215 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.625217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.625226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.625287 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.626018 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.626024 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.626025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.626029 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.626062 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.626080 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.626143 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.626204 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.626219 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260927 11:08:21.626466 22525 tablet_service.cc:1467] Processing CreateTablet for tablet c9e849a5c4c34ec6b4b22ceb7249f504 (DEFAULT_TABLE table=test-table [id=15eb37a3ce8f48f9a227bd0d8525dfe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:21.626782 22525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e849a5c4c34ec6b4b22ceb7249f504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:21.626785 22656 tablet_service.cc:1467] Processing CreateTablet for tablet c9e849a5c4c34ec6b4b22ceb7249f504 (DEFAULT_TABLE table=test-table [id=15eb37a3ce8f48f9a227bd0d8525dfe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:21.627034 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e849a5c4c34ec6b4b22ceb7249f504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:21.629428 22867 tablet_bootstrap.cc:492] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Bootstrap starting.
I20260927 11:08:21.630089 22869 tablet_bootstrap.cc:492] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: Bootstrap starting.
I20260927 11:08:21.630203 22867 tablet_bootstrap.cc:654] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:21.630255 22868 tablet_bootstrap.cc:492] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Bootstrap starting.
I20260927 11:08:21.630491 22867 log.cc:824] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:21.630730 22869 tablet_bootstrap.cc:654] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:21.630942 22868 tablet_bootstrap.cc:654] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:21.631094 22869 log.cc:824] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:21.631266 22867 tablet_bootstrap.cc:492] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: No bootstrap required, opened a new log
I20260927 11:08:21.631282 22868 log.cc:824] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:21.631340 22867 ts_tablet_manager.cc:1397] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:21.631824 22869 tablet_bootstrap.cc:492] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: No bootstrap required, opened a new log
I20260927 11:08:21.631894 22869 ts_tablet_manager.cc:1397] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:21.631969 22868 tablet_bootstrap.cc:492] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: No bootstrap required, opened a new log
I20260927 11:08:21.632043 22868 ts_tablet_manager.cc:1397] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:08:21.633741 22869 raft_consensus.cc:348] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.633874 22869 raft_consensus.cc:374] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:21.633795 22867 raft_consensus.cc:348] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.633912 22869 raft_consensus.cc:729] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323a39c92a8f49a19c0e6c6e2a14eda7, State: Initialized, Role: FOLLOWER
I20260927 11:08:21.633930 22867 raft_consensus.cc:374] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:21.633976 22867 raft_consensus.cc:729] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a8b8e1044945eeb001fc55607e8543, State: Initialized, Role: FOLLOWER
I20260927 11:08:21.634007 22869 consensus_queue.cc:260] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.634099 22867 consensus_queue.cc:260] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.634230 22869 ts_tablet_manager.cc:1428] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:08:21.634348 22867 ts_tablet_manager.cc:1428] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:08:21.634387 22591 heartbeater.cc:503] Master 127.17.95.254:42675 was elected leader, sending a full tablet report...
I20260927 11:08:21.634593 22853 heartbeater.cc:503] Master 127.17.95.254:42675 was elected leader, sending a full tablet report...
I20260927 11:08:21.634603 22868 raft_consensus.cc:348] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.634735 22868 raft_consensus.cc:374] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:21.634763 22868 raft_consensus.cc:729] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9c3e8bbbd745108ec082ce6efd7ce6, State: Initialized, Role: FOLLOWER
I20260927 11:08:21.634852 22868 consensus_queue.cc:260] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.635144 22868 ts_tablet_manager.cc:1428] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:08:21.635212 22722 heartbeater.cc:503] Master 127.17.95.254:42675 was elected leader, sending a full tablet report...
W20260927 11:08:21.637153 22592 tablet.cc:2369] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:08:21.748126 22723 tablet.cc:2369] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:21.766441 22875 raft_consensus.cc:482] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:21.766557 22875 raft_consensus.cc:504] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.766878 22875 leader_election.cc:288] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 323a39c92a8f49a19c0e6c6e2a14eda7 (127.17.95.193:34047), f4a8b8e1044945eeb001fc55607e8543 (127.17.95.195:41849)
I20260927 11:08:21.768173 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.767047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:34047 (local address 127.17.95.194:36095)
0927 11:08:21.767124 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.767136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.767173 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.767365 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.767366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.767368 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.767820 (+ 452us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.767821 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.767829 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.767842 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.768024 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.768025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.768026 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.768091 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.768106 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 11:08:21.768174 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.767173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:36095 (local address 127.17.95.193:34047)
0927 11:08:21.767232 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.767234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.767241 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:21.767264 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.767264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.767284 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.767350 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.767924 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.767925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.767925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.767929 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.767947 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.767952 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.768021 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.768096 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.768112 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 11:08:21.768275 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.767246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:51527 (local address 127.17.95.195:41849)
0927 11:08:21.767314 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.767316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.767495 (+ 179us) server_negotiation.cc:407] Connection header received
0927 11:08:21.767562 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.767563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.767567 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.767612 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.768069 (+ 457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.768070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.768070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.768075 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.768088 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.768092 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.768135 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.768196 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.768209 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 11:08:21.768296 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.767050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:41849 (local address 127.17.95.194:51527)
0927 11:08:21.767420 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.767429 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:21.767503 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:21.767614 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:21.767615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:21.767616 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:21.767991 (+ 375us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:21.767992 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:21.767998 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:21.768011 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:21.768144 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:21.768145 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:21.768145 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 11:08:21.768195 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 11:08:21.768208 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":190,"thread_start_us":114,"threads_started":1}
I20260927 11:08:21.768540 22545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e849a5c4c34ec6b4b22ceb7249f504" candidate_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" is_pre_election: true
I20260927 11:08:21.768707 22545 raft_consensus.cc:2454] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa9c3e8bbbd745108ec082ce6efd7ce6 in term 0.
I20260927 11:08:21.768796 22807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e849a5c4c34ec6b4b22ceb7249f504" candidate_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a8b8e1044945eeb001fc55607e8543" is_pre_election: true
I20260927 11:08:21.768924 22609 leader_election.cc:302] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 323a39c92a8f49a19c0e6c6e2a14eda7, aa9c3e8bbbd745108ec082ce6efd7ce6; no voters:
I20260927 11:08:21.768949 22807 raft_consensus.cc:2454] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa9c3e8bbbd745108ec082ce6efd7ce6 in term 0.
I20260927 11:08:21.769026 22875 raft_consensus.cc:2781] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:21.769080 22875 raft_consensus.cc:482] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:21.769107 22875 raft_consensus.cc:3037] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.769798 22875 raft_consensus.cc:504] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.769932 22875 leader_election.cc:288] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [CANDIDATE]: Term 1 election: Requested vote from peers 323a39c92a8f49a19c0e6c6e2a14eda7 (127.17.95.193:34047), f4a8b8e1044945eeb001fc55607e8543 (127.17.95.195:41849)
I20260927 11:08:21.770066 22807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e849a5c4c34ec6b4b22ceb7249f504" candidate_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a8b8e1044945eeb001fc55607e8543"
I20260927 11:08:21.770062 22545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e849a5c4c34ec6b4b22ceb7249f504" candidate_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7"
I20260927 11:08:21.770135 22807 raft_consensus.cc:3037] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.770135 22545 raft_consensus.cc:3037] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.770975 22807 raft_consensus.cc:2454] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa9c3e8bbbd745108ec082ce6efd7ce6 in term 1.
I20260927 11:08:21.770975 22545 raft_consensus.cc:2454] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa9c3e8bbbd745108ec082ce6efd7ce6 in term 1.
I20260927 11:08:21.771183 22610 leader_election.cc:302] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa9c3e8bbbd745108ec082ce6efd7ce6, f4a8b8e1044945eeb001fc55607e8543; no voters:
I20260927 11:08:21.771272 22875 raft_consensus.cc:2781] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:21.771411 22875 raft_consensus.cc:686] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 1 LEADER]: Becoming Leader. State: Replica: aa9c3e8bbbd745108ec082ce6efd7ce6, State: Running, Role: LEADER
I20260927 11:08:21.771497 22875 consensus_queue.cc:237] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } }
I20260927 11:08:21.772168 22402 catalog_manager.cc:5495] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa9c3e8bbbd745108ec082ce6efd7ce6 (127.17.95.194). New cstate: current_term: 1 leader_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa9c3e8bbbd745108ec082ce6efd7ce6" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42273 } health_report { overall_health: HEALTHY } } }
I20260927 11:08:21.806281 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.805241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.17.95.194:42273)
0927 11:08:21.805303 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.805305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.805333 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:08:21.805379 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.805380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.805393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.805490 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.806045 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.806047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.806047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.806052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.806068 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.806072 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.806134 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.806190 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.806204 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:21.808074 22807 raft_consensus.cc:1261] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 1 FOLLOWER]: Refusing update from remote peer aa9c3e8bbbd745108ec082ce6efd7ce6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:21.808075 22545 raft_consensus.cc:1261] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 1 FOLLOWER]: Refusing update from remote peer aa9c3e8bbbd745108ec082ce6efd7ce6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:21.808445 22875 consensus_queue.cc:1035] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4a8b8e1044945eeb001fc55607e8543" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:21.808609 22877 consensus_queue.cc:1035] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "323a39c92a8f49a19c0e6c6e2a14eda7" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 34047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:21.811033 22880 mvcc.cc:205] Tried to move back new op lower bound from 7333917908203302912 to 7333917908056444928. Current Snapshot: MvccSnapshot[applied={T|T < 7333917908203302912}]
I20260927 11:08:21.812014 22881 mvcc.cc:205] Tried to move back new op lower bound from 7333917908203302912 to 7333917908056444928. Current Snapshot: MvccSnapshot[applied={T|T < 7333917908203302912 or (T in {7333917908203302912})}]
I20260927 11:08:21.813127 22403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:21.813227 22403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:21.813958 22403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 463e66d315fd444faa3c9aae6ac10cd6: Sending DeleteTablet for 3 replicas of tablet c9e849a5c4c34ec6b4b22ceb7249f504
I20260927 11:08:21.814250 22525 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e849a5c4c34ec6b4b22ceb7249f504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:21 UTC) from {username='slave'} at 127.0.0.1:37928
I20260927 11:08:21.814361 22787 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e849a5c4c34ec6b4b22ceb7249f504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:21 UTC) from {username='slave'} at 127.0.0.1:32998
I20260927 11:08:21.814378 22656 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e849a5c4c34ec6b4b22ceb7249f504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:21 UTC) from {username='slave'} at 127.0.0.1:37342
I20260927 11:08:21.814616 22891 tablet_replica.cc:330] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7: stopping tablet replica
I20260927 11:08:21.814630 22893 tablet_replica.cc:330] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: stopping tablet replica
I20260927 11:08:21.814710 22893 raft_consensus.cc:2229] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:21.814725 22891 raft_consensus.cc:2229] T c9e849a5c4c34ec6b4b22ceb7249f504 P 323a39c92a8f49a19c0e6c6e2a14eda7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:21.814739 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22463
I20260927 11:08:21.814800 22893 raft_consensus.cc:2258] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:21.814937 22892 tablet_replica.cc:330] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: stopping tablet replica
I20260927 11:08:21.815025 22892 raft_consensus.cc:2229] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:21.815152 22892 raft_consensus.cc:2258] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:21.815423 22893 ts_tablet_manager.cc:1905] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:21.816910 22893 ts_tablet_manager.cc:1918] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:21.817022 22893 log.cc:1196] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/c9e849a5c4c34ec6b4b22ceb7249f504
I20260927 11:08:21.817245 22893 ts_tablet_manager.cc:1939] T c9e849a5c4c34ec6b4b22ceb7249f504 P aa9c3e8bbbd745108ec082ce6efd7ce6: Deleting consensus metadata
I20260927 11:08:21.817588 22388 catalog_manager.cc:4841] TS aa9c3e8bbbd745108ec082ce6efd7ce6 (127.17.95.194:42273): tablet c9e849a5c4c34ec6b4b22ceb7249f504 (table test-table [id=15eb37a3ce8f48f9a227bd0d8525dfe5]) successfully deleted
I20260927 11:08:21.817679 22892 ts_tablet_manager.cc:1905] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:21.818838 22892 ts_tablet_manager.cc:1918] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:21.818915 22892 log.cc:1196] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/c9e849a5c4c34ec6b4b22ceb7249f504
I20260927 11:08:21.819190 22892 ts_tablet_manager.cc:1939] T c9e849a5c4c34ec6b4b22ceb7249f504 P f4a8b8e1044945eeb001fc55607e8543: Deleting consensus metadata
I20260927 11:08:21.819561 22388 catalog_manager.cc:4841] TS f4a8b8e1044945eeb001fc55607e8543 (127.17.95.195:41849): tablet c9e849a5c4c34ec6b4b22ceb7249f504 (table test-table [id=15eb37a3ce8f48f9a227bd0d8525dfe5]) successfully deleted
W20260927 11:08:21.823237 22387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:34047 (error 108)
I20260927 11:08:21.823418 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22594
I20260927 11:08:21.824015 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.823516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:34047 (local address 127.0.0.1:37934)
0927 11:08:21.823903 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:21.823938 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:34047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371}
W20260927 11:08:21.824160 22387 catalog_manager.cc:4584] TS 323a39c92a8f49a19c0e6c6e2a14eda7 (127.17.95.193:34047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9e849a5c4c34ec6b4b22ceb7249f504: Network error: Client connection negotiation failed: client connection to 127.17.95.193:34047: connect: Connection refused (error 111)
I20260927 11:08:21.828402 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22725
I20260927 11:08:21.832540 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22372
2026-09-27T11:08:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (705 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 11:08:21.851423 17791 mini_oidc.cc:152] Starting JWKS server
I20260927 11:08:21.852357 17791 webserver.cc:460] Webserver started at https://127.0.0.1:37997/ using document root <none> and password file <none>
I20260927 11:08:21.852429 17791 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 11:08:21.852574 17791 webserver.cc:460] Webserver started at http://127.0.0.1:45019/ using document root <none> and password file <none>
2026-09-27T11:08:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:21Z Disabled control of system clock
I20260927 11:08:21.859333 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40783
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40783
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37997/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/testchainca.cert with env {}
W20260927 11:08:21.936836 22900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:21.937075 22900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:21.937112 22900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:21.938220 22900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:21.938273 22900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:21.938298 22900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:21.938326 22900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:21.938347 22900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 11:08:21.938373 22900 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 11:08:21.938400 22900 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37997/jwks.json
I20260927 11:08:21.939930 22900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40783
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37997/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:21.940188 22900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:21.940390 22900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:21.944978 22905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.944978 22906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:21.944990 22908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:21.945114 22900 server_base.cc:1034] running on GCE node
I20260927 11:08:21.945336 22900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:21.945619 22900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:21.946776 22900 hybrid_clock.cc:648] HybridClock initialized: now 1790507301946753 us; error 48 us; skew 500 ppm
I20260927 11:08:21.948032 22900 webserver.cc:460] Webserver started at http://127.17.95.254:46297/ using document root <none> and password file <none>
I20260927 11:08:21.948271 22900 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:21.948333 22900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:21.948467 22900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:21.949466 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "3ffc5fe53fb740cc8ef46088efc1bfad"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "f9f9031487430e2a5eeb9df71d497152"
server_key_iv: "c5aba0ebb2e23a674c4601267adaa54b"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.949831 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "3ffc5fe53fb740cc8ef46088efc1bfad"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "f9f9031487430e2a5eeb9df71d497152"
server_key_iv: "c5aba0ebb2e23a674c4601267adaa54b"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.951130 22900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:21.951792 22914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:21.951963 22900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:21.952057 22900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "3ffc5fe53fb740cc8ef46088efc1bfad"
format_stamp: "Formatted at 2026-09-27 11:08:21 on dist-test-slave-6j0v"
server_key: "f9f9031487430e2a5eeb9df71d497152"
server_key_iv: "c5aba0ebb2e23a674c4601267adaa54b"
server_key_version: "encryptionkey@0"
I20260927 11:08:21.952135 22900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:21.964174 22900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:21.964447 22900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:21.964588 22900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:21.968571 22900 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:40783
I20260927 11:08:21.968608 22966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:40783 every 8 connection(s)
I20260927 11:08:21.968897 22900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:21.969738 22967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:21.971909 22967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad: Bootstrap starting.
I20260927 11:08:21.972443 22967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:21.972726 22967 log.cc:824] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:21.973196 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 22900
I20260927 11:08:21.973325 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:21.973443 22967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad: No bootstrap required, opened a new log
I20260927 11:08:21.973503 17791 external_mini_cluster.cc:1428] Setting key d3d3293ead69240074c1b7dd37635b78
I20260927 11:08:21.975348 22967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40783 } }
I20260927 11:08:21.975493 22967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:21.975517 22967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ffc5fe53fb740cc8ef46088efc1bfad, State: Initialized, Role: FOLLOWER
I20260927 11:08:21.975651 22967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40783 } }
I20260927 11:08:21.975706 22967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:21.975741 22967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:21.975787 22967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:21.976325 22967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40783 } }
I20260927 11:08:21.976408 22967 leader_election.cc:302] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ffc5fe53fb740cc8ef46088efc1bfad; no voters:
I20260927 11:08:21.976614 22967 leader_election.cc:288] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:21.976677 22972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:21.976820 22972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [term 1 LEADER]: Becoming Leader. State: Replica: 3ffc5fe53fb740cc8ef46088efc1bfad, State: Running, Role: LEADER
I20260927 11:08:21.976853 22967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:21.976902 22972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40783 } }
I20260927 11:08:21.977202 22972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ffc5fe53fb740cc8ef46088efc1bfad. Latest consensus state: current_term: 1 leader_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40783 } } }
I20260927 11:08:21.977293 22972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:21.977217 22973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffc5fe53fb740cc8ef46088efc1bfad" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40783 } } }
I20260927 11:08:21.977421 22973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:21.977624 22979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:21.978358 22979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:21.980006 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:21.973961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.17.95.254:40783)
0927 11:08:21.974192 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:21.974199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:21.974222 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:08:21.974636 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:21.974642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:21.974694 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:21.974830 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:21.975054 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.976396 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.977634 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:21.978304 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:21.978414 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:21.979338 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:21.979341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:21.979341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:21.979545 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:21.979588 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:21.979599 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:21.979674 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:21.979751 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 11:08:21.979826 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260927 11:08:21.980917 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:08:21.983446 22979 catalog_manager.cc:1347] Generated new cluster ID: e80013ce88df4a7c9136229d836c7de2
I20260927 11:08:21.983505 22979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:21.998790 22979 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:21.999470 22979 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:22.002465 22979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ffc5fe53fb740cc8ef46088efc1bfad: Generated new TSK 0
I20260927 11:08:22.002626 22979 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 11:08:22.022753 22997 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 11:08:22.022850 22997 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 11:08:22.023123 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.008615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.17.95.254:40783)
0927 11:08:22.008658 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.008660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.008764 (+ 104us) server_negotiation.cc:407] Connection header received
0927 11:08:22.008866 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.008867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.008878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.008941 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0927 11:08:22.009116 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.009593 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.010355 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.010576 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.010649 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.010850 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 11:08:22.022866 (+ 12016us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 11:08:22.023038 (+ 172us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59854: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 11:08:22.023198 22971 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59854: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 11:08:22.023438 22996 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:22.008547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40783 (local address 127.0.0.1:59854)
0927 11:08:22.008691 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:22.008704 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:22.008776 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:22.008946 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:22.008947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:22.008948 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0927 11:08:22.009038 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.009040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.009672 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.009673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.010251 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.010253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.010658 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.010659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.010732 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.010748 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 11:08:22.023008 (+ 12260us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 11:08:22.023148 (+ 140us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.95.254:40783: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.95.254:40783: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37997/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 11:08:22.023833 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 22900
2026-09-27T11:08:22Z chronyd exiting
I20260927 11:08:22.059500 17791 test_util.cc:175] -----------------------------------------------
I20260927 11:08:22.059568 17791 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790507277271108-17791-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (209 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 11:08:22.060297 17791 mini_oidc.cc:152] Starting JWKS server
I20260927 11:08:22.060987 17791 webserver.cc:460] Webserver started at https://127.0.0.1:43185/ using document root <none> and password file <none>
I20260927 11:08:22.061026 17791 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 11:08:22.061173 17791 webserver.cc:460] Webserver started at http://127.0.0.1:39969/ using document root <none> and password file <none>
2026-09-27T11:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:22Z Disabled control of system clock
I20260927 11:08:22.068933 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39309
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:37061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:39309
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43185/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/testchainca.cert with env {}
W20260927 11:08:22.147189 23003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:22.147414 23003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:22.147436 23003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:22.148602 23003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:22.148640 23003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:22.148653 23003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:22.148690 23003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:22.148716 23003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 11:08:22.148736 23003 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 11:08:22.148761 23003 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43185/jwks.json
I20260927 11:08:22.150166 23003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:39309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43185/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:22.150393 23003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:22.150589 23003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:22.154994 23011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:22.155009 23009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:22.155153 23003 server_base.cc:1034] running on GCE node
W20260927 11:08:22.155011 23008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:22.155393 23003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:22.155675 23003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:22.156834 23003 hybrid_clock.cc:648] HybridClock initialized: now 1790507302156819 us; error 40 us; skew 500 ppm
I20260927 11:08:22.157994 23003 webserver.cc:460] Webserver started at http://127.17.95.254:45497/ using document root <none> and password file <none>
I20260927 11:08:22.158245 23003 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:22.158308 23003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:22.158429 23003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:22.159480 23003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "d01001ac4b994eb695dae1fdf299a261"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "5ce11b92b3e1f02a75cad1de27d8638a"
server_key_iv: "8ba57bd311b0840d623afae2aa896a74"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.159822 23003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "d01001ac4b994eb695dae1fdf299a261"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "5ce11b92b3e1f02a75cad1de27d8638a"
server_key_iv: "8ba57bd311b0840d623afae2aa896a74"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.161067 23003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:22.161751 23017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:22.161901 23003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:22.162006 23003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "d01001ac4b994eb695dae1fdf299a261"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "5ce11b92b3e1f02a75cad1de27d8638a"
server_key_iv: "8ba57bd311b0840d623afae2aa896a74"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.162082 23003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:22.196636 23003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:22.196926 23003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:22.197093 23003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:22.200592 23003 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:39309
I20260927 11:08:22.200631 23069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:39309 every 8 connection(s)
I20260927 11:08:22.200963 23003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:22.201589 23070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:22.203210 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23003
I20260927 11:08:22.203310 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:22.203518 17791 external_mini_cluster.cc:1428] Setting key 76cb31b899cbda005fe0fbf40df249a0
I20260927 11:08:22.204583 23070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261: Bootstrap starting.
I20260927 11:08:22.205272 23070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:22.205601 23070 log.cc:824] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:22.206310 23070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261: No bootstrap required, opened a new log
I20260927 11:08:22.208439 23070 raft_consensus.cc:348] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01001ac4b994eb695dae1fdf299a261" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39309 } }
I20260927 11:08:22.208570 23070 raft_consensus.cc:374] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:22.208591 23070 raft_consensus.cc:729] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d01001ac4b994eb695dae1fdf299a261, State: Initialized, Role: FOLLOWER
I20260927 11:08:22.208741 23070 consensus_queue.cc:260] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01001ac4b994eb695dae1fdf299a261" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39309 } }
I20260927 11:08:22.208828 23070 raft_consensus.cc:388] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:22.208863 23070 raft_consensus.cc:482] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:22.208911 23070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:22.209374 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.204037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.17.95.254:39309)
0927 11:08:22.204248 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.204257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.204289 (+ 32us) server_negotiation.cc:407] Connection header received
0927 11:08:22.204776 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.204784 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.204862 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.205039 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:22.205198 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.206504 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.207316 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.207923 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.208070 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.208708 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:22.208712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:22.208712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:22.208923 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:22.208975 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:22.208989 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:22.209056 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:22.209136 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 11:08:22.209212 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260927 11:08:22.209534 23070 raft_consensus.cc:504] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01001ac4b994eb695dae1fdf299a261" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39309 } }
I20260927 11:08:22.209635 23070 leader_election.cc:302] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d01001ac4b994eb695dae1fdf299a261; no voters:
I20260927 11:08:22.209777 23070 leader_election.cc:288] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:22.209846 23075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:22.210001 23075 raft_consensus.cc:686] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [term 1 LEADER]: Becoming Leader. State: Replica: d01001ac4b994eb695dae1fdf299a261, State: Running, Role: LEADER
I20260927 11:08:22.210017 23070 sys_catalog.cc:564] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:22.210142 23075 consensus_queue.cc:237] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01001ac4b994eb695dae1fdf299a261" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39309 } }
I20260927 11:08:22.210269 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:08:22.210450 23077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d01001ac4b994eb695dae1fdf299a261. Latest consensus state: current_term: 1 leader_uuid: "d01001ac4b994eb695dae1fdf299a261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01001ac4b994eb695dae1fdf299a261" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39309 } } }
I20260927 11:08:22.210577 23077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:22.210834 23083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:22.211321 23083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:22.211740 23076 sys_catalog.cc:455] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d01001ac4b994eb695dae1fdf299a261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01001ac4b994eb695dae1fdf299a261" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39309 } } }
I20260927 11:08:22.211840 23076 sys_catalog.cc:458] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:22.212761 23083 catalog_manager.cc:1347] Generated new cluster ID: 0b46e2d7f13648dc83e0a5408decff0a
I20260927 11:08:22.212826 23083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:22.222618 23083 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:22.223155 23083 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:22.226310 23083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d01001ac4b994eb695dae1fdf299a261: Generated new TSK 0
I20260927 11:08:22.226470 23083 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 11:08:22.254431 23100 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 11:08:22.254498 23100 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 11:08:22.254733 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.232671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36550 (local address 127.17.95.254:39309)
0927 11:08:22.232729 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.232732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.232853 (+ 121us) server_negotiation.cc:407] Connection header received
0927 11:08:22.232920 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.232921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.232932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.232997 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0927 11:08:22.233153 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.233643 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.234787 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.234979 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.235037 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.235220 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 11:08:22.254550 (+ 19330us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43185/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 11:08:22.254687 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36550: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43185/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":35}
W20260927 11:08:22.254794 23073 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36550: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43185/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260927 11:08:22.254840 23099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:22.232639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39309 (local address 127.0.0.1:36550)
0927 11:08:22.232784 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:22.232796 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:22.232858 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:22.233007 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:22.233007 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:22.233009 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 11:08:22.233088 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.233089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.233716 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.233717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.234710 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.234713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.235057 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.235058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.235132 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.235138 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 11:08:22.254651 (+ 19513us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43185/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 11:08:22.254747 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.95.254:39309: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43185/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260927 11:08:22.255141 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23003
2026-09-27T11:08:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 11:08:22.292469 17791 mini_oidc.cc:152] Starting JWKS server
I20260927 11:08:22.293115 17791 webserver.cc:460] Webserver started at https://127.0.0.1:43315/ using document root <none> and password file <none>
I20260927 11:08:22.293174 17791 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 11:08:22.293340 17791 webserver.cc:460] Webserver started at http://127.0.0.1:34981/ using document root <none> and password file <none>
2026-09-27T11:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:22Z Disabled control of system clock
I20260927 11:08:22.300429 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35549
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:38323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:35549
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43315/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 11:08:22.380486 23106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:22.380699 23106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:22.380721 23106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:22.381896 23106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:22.381934 23106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:22.381947 23106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:22.381959 23106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:22.382007 23106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 11:08:22.382030 23106 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 11:08:22.382052 23106 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43315/jwks.json
I20260927 11:08:22.383395 23106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:38323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:35549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43315/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:22.383622 23106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:22.383885 23106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:08:22.388478 23106 server_base.cc:1034] running on GCE node
W20260927 11:08:22.388446 23112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:22.388432 23114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:22.388432 23111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:22.388866 23106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:22.389133 23106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:22.390275 23106 hybrid_clock.cc:648] HybridClock initialized: now 1790507302390261 us; error 29 us; skew 500 ppm
I20260927 11:08:22.391480 23106 webserver.cc:460] Webserver started at http://127.17.95.254:44311/ using document root <none> and password file <none>
I20260927 11:08:22.391711 23106 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:22.391754 23106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:22.391885 23106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:22.392994 23106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "d84f06cf354d4a2bba322620d5e0eb8d"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "4bc8e622e4e2c8adbbe14e3507ca98d7"
server_key_iv: "1788351bda6b037e4704fd386ce91179"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.393338 23106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "d84f06cf354d4a2bba322620d5e0eb8d"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "4bc8e622e4e2c8adbbe14e3507ca98d7"
server_key_iv: "1788351bda6b037e4704fd386ce91179"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.394596 23106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:08:22.395298 23120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:22.395469 23106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:22.395570 23106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "d84f06cf354d4a2bba322620d5e0eb8d"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "4bc8e622e4e2c8adbbe14e3507ca98d7"
server_key_iv: "1788351bda6b037e4704fd386ce91179"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.395638 23106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:22.415146 23106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:22.415395 23106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:22.415539 23106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:22.419167 23106 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:35549
I20260927 11:08:22.419221 23172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:35549 every 8 connection(s)
I20260927 11:08:22.419503 23106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:22.420102 23173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:22.422685 23173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d: Bootstrap starting.
I20260927 11:08:22.423272 23173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:22.423552 23173 log.cc:824] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:22.424141 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23106
I20260927 11:08:22.424206 23173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d: No bootstrap required, opened a new log
I20260927 11:08:22.424223 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:22.424424 17791 external_mini_cluster.cc:1428] Setting key 61e2cc08cec8e28791cb641f2de0b2fd
I20260927 11:08:22.426313 23173 raft_consensus.cc:348] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35549 } }
I20260927 11:08:22.426456 23173 raft_consensus.cc:374] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:22.426501 23173 raft_consensus.cc:729] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84f06cf354d4a2bba322620d5e0eb8d, State: Initialized, Role: FOLLOWER
I20260927 11:08:22.426630 23173 consensus_queue.cc:260] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35549 } }
I20260927 11:08:22.426718 23173 raft_consensus.cc:388] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:22.426771 23173 raft_consensus.cc:482] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:22.426821 23173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:22.427410 23173 raft_consensus.cc:504] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35549 } }
I20260927 11:08:22.427516 23173 leader_election.cc:302] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d84f06cf354d4a2bba322620d5e0eb8d; no voters:
I20260927 11:08:22.427671 23173 leader_election.cc:288] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:22.427714 23178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:22.427845 23178 raft_consensus.cc:686] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [term 1 LEADER]: Becoming Leader. State: Replica: d84f06cf354d4a2bba322620d5e0eb8d, State: Running, Role: LEADER
I20260927 11:08:22.427897 23173 sys_catalog.cc:564] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:22.427950 23178 consensus_queue.cc:237] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35549 } }
I20260927 11:08:22.428267 23179 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35549 } } }
I20260927 11:08:22.428260 23178 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84f06cf354d4a2bba322620d5e0eb8d. Latest consensus state: current_term: 1 leader_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f06cf354d4a2bba322620d5e0eb8d" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35549 } } }
I20260927 11:08:22.428434 23178 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:22.428416 23179 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:22.429217 23187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:22.429858 23187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:22.430846 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.424878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.17.95.254:35549)
0927 11:08:22.425061 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.425068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.425092 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:08:22.425493 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.425505 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.425559 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.425665 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:22.425909 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.427345 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.428448 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.429123 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.429222 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.430130 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:22.430134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:22.430134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:22.430316 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:22.430356 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:22.430369 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:22.430404 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:22.430549 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 11:08:22.430640 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260927 11:08:22.431663 23187 catalog_manager.cc:1347] Generated new cluster ID: 6ab45263fcba4d0b9d374b3f4d0e6aca
I20260927 11:08:22.431717 23187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:22.431775 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:08:22.445082 23187 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:22.445559 23187 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:22.452764 23187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d84f06cf354d4a2bba322620d5e0eb8d: Generated new TSK 0
I20260927 11:08:22.452925 23187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:22.462360 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.458994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.17.95.254:35549)
0927 11:08:22.459040 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.459042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.459133 (+ 91us) server_negotiation.cc:407] Connection header received
0927 11:08:22.459214 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.459215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.459225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.459280 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0927 11:08:22.459413 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.459895 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.461010 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.461201 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.461253 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.461423 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 11:08:22.462136 (+ 713us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 11:08:22.462273 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38036: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 11:08:22.462407 23177 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38036: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260927 11:08:22.462435 23202 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:22.458902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:35549 (local address 127.0.0.1:38036)
0927 11:08:22.459054 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:22.459086 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:22.459149 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:22.459299 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:22.459300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:22.459302 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 11:08:22.459354 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.459356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.459966 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.459967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.460931 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.460933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.461279 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.461280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.461339 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.461344 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 11:08:22.462246 (+ 902us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 11:08:22.462322 (+ 76us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.95.254:35549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43315/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260927 11:08:22.462976 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23106
E20260927 11:08:22.474967 23203 webserver.cc:533] Webserver: sslize failed:
E20260927 11:08:22.475020 23203 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-27T11:08:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (209 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T11:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:22Z Disabled control of system clock
I20260927 11:08:22.510150 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35807
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:35807
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/testchainca.cert with env {}
W20260927 11:08:22.587721 23207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:22.587949 23207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:22.587986 23207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:22.589107 23207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:22.589181 23207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:22.589208 23207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:22.589298 23207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:22.589359 23207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 11:08:22.589385 23207 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 11:08:22.589416 23207 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 11:08:22.590991 23207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:35807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:22.591269 23207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:22.591478 23207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:22.595979 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:22.595988 23215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:22.595991 23212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:22.596230 23207 server_base.cc:1034] running on GCE node
I20260927 11:08:22.596436 23207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:22.596691 23207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:22.597857 23207 hybrid_clock.cc:648] HybridClock initialized: now 1790507302597842 us; error 40 us; skew 500 ppm
I20260927 11:08:22.598969 23207 webserver.cc:460] Webserver started at http://127.17.95.254:44927/ using document root <none> and password file <none>
I20260927 11:08:22.599213 23207 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:22.599254 23207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:22.599396 23207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:22.600406 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "3df3b0474ef540b2afe7c4aced03953f"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "7e17c38a83db0d97cffdda602478a719"
server_key_iv: "9d7696e000039b86a7b499fcc2c57228"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.600777 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "3df3b0474ef540b2afe7c4aced03953f"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "7e17c38a83db0d97cffdda602478a719"
server_key_iv: "9d7696e000039b86a7b499fcc2c57228"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.602029 23207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:22.602736 23221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:22.602886 23207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:22.602967 23207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "3df3b0474ef540b2afe7c4aced03953f"
format_stamp: "Formatted at 2026-09-27 11:08:22 on dist-test-slave-6j0v"
server_key: "7e17c38a83db0d97cffdda602478a719"
server_key_iv: "9d7696e000039b86a7b499fcc2c57228"
server_key_version: "encryptionkey@0"
I20260927 11:08:22.603044 23207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:22.614918 23207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:22.615262 23207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:22.615401 23207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:22.618924 23207 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:35807
I20260927 11:08:22.618955 23273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:35807 every 8 connection(s)
I20260927 11:08:22.619283 23207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:22.620131 23274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:22.622092 23274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f: Bootstrap starting.
I20260927 11:08:22.622603 23274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:22.622828 23274 log.cc:824] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:22.623555 23274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f: No bootstrap required, opened a new log
I20260927 11:08:22.624488 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23207
I20260927 11:08:22.624606 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:22.624779 17791 external_mini_cluster.cc:1428] Setting key 543de9a0a9f127bde5d7f04a0e528d33
I20260927 11:08:22.625334 23274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df3b0474ef540b2afe7c4aced03953f" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35807 } }
I20260927 11:08:22.625502 23274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:22.625560 23274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df3b0474ef540b2afe7c4aced03953f, State: Initialized, Role: FOLLOWER
I20260927 11:08:22.625679 23274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df3b0474ef540b2afe7c4aced03953f" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35807 } }
I20260927 11:08:22.625746 23274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:22.625765 23274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:22.625826 23274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:22.626333 23274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df3b0474ef540b2afe7c4aced03953f" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35807 } }
I20260927 11:08:22.626444 23274 leader_election.cc:302] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3df3b0474ef540b2afe7c4aced03953f; no voters:
I20260927 11:08:22.626598 23274 leader_election.cc:288] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:22.626647 23279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:22.626770 23279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [term 1 LEADER]: Becoming Leader. State: Replica: 3df3b0474ef540b2afe7c4aced03953f, State: Running, Role: LEADER
I20260927 11:08:22.626842 23274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:22.626844 23279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df3b0474ef540b2afe7c4aced03953f" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35807 } }
I20260927 11:08:22.627210 23279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3df3b0474ef540b2afe7c4aced03953f. Latest consensus state: current_term: 1 leader_uuid: "3df3b0474ef540b2afe7c4aced03953f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df3b0474ef540b2afe7c4aced03953f" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35807 } } }
I20260927 11:08:22.627300 23279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:22.627224 23281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3df3b0474ef540b2afe7c4aced03953f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df3b0474ef540b2afe7c4aced03953f" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35807 } } }
I20260927 11:08:22.627441 23281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:22.627506 23285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:22.628041 23285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:22.630982 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.625306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.17.95.254:35807)
0927 11:08:22.625512 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.625521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.625551 (+ 30us) server_negotiation.cc:407] Connection header received
0927 11:08:22.625675 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.625681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.625735 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.625907 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:22.626207 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.627844 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.629043 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.629606 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.629724 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.630304 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:22.630307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:22.630308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:22.630539 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:22.630576 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:22.630591 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:22.630657 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:22.630716 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 11:08:22.630787 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260927 11:08:22.631174 23285 catalog_manager.cc:1347] Generated new cluster ID: c1203a06a64947e6880449c0daafb811
I20260927 11:08:22.631227 23285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:22.631834 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:08:22.651139 23285 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:22.651659 23285 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:22.657675 23285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3df3b0474ef540b2afe7c4aced03953f: Generated new TSK 0
I20260927 11:08:22.657845 23285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:23.149624 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:22.671060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45776 (local address 127.17.95.254:35807)
0927 11:08:22.671126 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:22.671128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:22.671248 (+ 120us) server_negotiation.cc:407] Connection header received
0927 11:08:22.671312 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:22.671313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:22.671323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:22.671374 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0927 11:08:22.671520 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.671969 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.673051 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.673218 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.673260 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.673424 (+ 164us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 11:08:23.149409 (+475985us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 11:08:23.149561 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45776: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260927 11:08:23.149698 23278 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45776: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 11:08:23.149873 23303 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:22.671000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:35807 (local address 127.0.0.1:45776)
0927 11:08:22.671173 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:22.671184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:22.671246 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:22.671382 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:22.671383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:22.671385 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 11:08:22.671454 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.671456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.672056 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.672057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.672970 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:22.672972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:22.673280 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:22.673282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:22.673341 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:22.673346 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 11:08:23.149568 (+476222us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 11:08:23.149680 (+ 112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.95.254:35807: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
I20260927 11:08:23.150239 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23207
2026-09-27T11:08:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (666 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:23 dist-test-slave-6j0v krb5kdc[23315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:23 dist-test-slave-6j0v krb5kdc[23315](info): set up 2 sockets
Sep 27 11:08:23 dist-test-slave-6j0v krb5kdc[23315](info): commencing operation
krb5kdc: starting...
W20260927 11:08:24.335263 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:24.444339 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40615
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:24.524847 23328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:24.525049 23328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:24.525087 23328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:24.526207 23328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:24.526273 23328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:24.526301 23328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:24.526335 23328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:24.526367 23328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:24.527868 23328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:24.528127 23328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:24.528342 23328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:24.532908 23334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:24.532968 23333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:24.532915 23336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.533088 23328 server_base.cc:1034] running on GCE node
I20260927 11:08:24.533301 23328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:24.533574 23328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:24.534749 23328 hybrid_clock.cc:648] HybridClock initialized: now 1790507304534742 us; error 38 us; skew 500 ppm
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:24.537252 23328 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:24.537671 23328 webserver.cc:460] Webserver started at http://127.17.95.254:42195/ using document root <none> and password file <none>
I20260927 11:08:24.537885 23328 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:24.537946 23328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:24.538074 23328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:24.538985 23328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "6eb6306c3df246ec8b90f74f91e58518"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "9e3d3de8166ee21aef76977395878864"
server_key_iv: "2fb8541d216377f587920111f187c09d"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.539330 23328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "6eb6306c3df246ec8b90f74f91e58518"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "9e3d3de8166ee21aef76977395878864"
server_key_iv: "2fb8541d216377f587920111f187c09d"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.540455 23328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:24.541059 23343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.541211 23328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.541285 23328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "6eb6306c3df246ec8b90f74f91e58518"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "9e3d3de8166ee21aef76977395878864"
server_key_iv: "2fb8541d216377f587920111f187c09d"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.541371 23328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:24.549655 23328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:24.551652 23328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:24.551795 23328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:24.555424 23328 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:40615
I20260927 11:08:24.555450 23395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:40615 every 8 connection(s)
I20260927 11:08:24.555756 23328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:24.556520 23396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:24.558344 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23328
I20260927 11:08:24.558480 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:24.558521 23396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518: Bootstrap starting.
I20260927 11:08:24.558694 17791 external_mini_cluster.cc:1428] Setting key b41717c23c44c830c55cbd59bfada24e
I20260927 11:08:24.559191 23396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:24.559520 23396 log.cc:824] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:24.560204 23396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518: No bootstrap required, opened a new log
I20260927 11:08:24.562413 23396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb6306c3df246ec8b90f74f91e58518" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40615 } }
I20260927 11:08:24.562553 23396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:24.562575 23396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb6306c3df246ec8b90f74f91e58518, State: Initialized, Role: FOLLOWER
I20260927 11:08:24.562711 23396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb6306c3df246ec8b90f74f91e58518" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40615 } }
I20260927 11:08:24.562791 23396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:24.562811 23396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:24.562868 23396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:24.563509 23396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb6306c3df246ec8b90f74f91e58518" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40615 } }
I20260927 11:08:24.563608 23396 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eb6306c3df246ec8b90f74f91e58518; no voters:
I20260927 11:08:24.563758 23396 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:24.563922 23402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:24.564009 23396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:24.564371 23402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb6306c3df246ec8b90f74f91e58518, State: Running, Role: LEADER
I20260927 11:08:24.564494 23402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb6306c3df246ec8b90f74f91e58518" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40615 } }
I20260927 11:08:24.564841 23403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb6306c3df246ec8b90f74f91e58518. Latest consensus state: current_term: 1 leader_uuid: "6eb6306c3df246ec8b90f74f91e58518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb6306c3df246ec8b90f74f91e58518" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40615 } } }
I20260927 11:08:24.564833 23401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eb6306c3df246ec8b90f74f91e58518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb6306c3df246ec8b90f74f91e58518" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40615 } } }
I20260927 11:08:24.564926 23403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:24.564930 23401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:24.565137 23411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:24.565584 23411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:24.567373 23411 catalog_manager.cc:1347] Generated new cluster ID: 6e14506d456a4bf3bf379c50374088cb
I20260927 11:08:24.567428 23411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:24.570632 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:24.559179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.17.95.254:40615)
0927 11:08:24.559370 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:24.559382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:24.559413 (+ 31us) server_negotiation.cc:407] Connection header received
0927 11:08:24.559875 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:24.559882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:24.559956 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:24.560102 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:24.560741 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.562129 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.563106 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.563704 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.563814 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:24.566678 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:24.566681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:24.566682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:24.566692 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:24.568709 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:24.569117 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:24.569120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:24.569120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:24.569212 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:24.569420 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:24.569421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:24.569421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:24.569691 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:24.570240 (+ 549us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:24.570309 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 11:08:24.570434 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260927 11:08:24.573043 23411 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:24.573534 23411 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:24.579526 23411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb6306c3df246ec8b90f74f91e58518: Generated new TSK 0
I20260927 11:08:24.579675 23411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:24.605523 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:40615
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:24.685251 23422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:24.685537 23422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:24.685590 23422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:24.687286 23422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:24.687328 23422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:24.687355 23422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:24.688751 23422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:40615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:24.688922 23422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:24.689093 23422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:24.693668 23427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.693710 23422 server_base.cc:1034] running on GCE node
W20260927 11:08:24.693706 23430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:24.693683 23428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.694096 23422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:24.694278 23422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:24.695402 23422 hybrid_clock.cc:648] HybridClock initialized: now 1790507304695391 us; error 27 us; skew 500 ppm
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:24.697856 23422 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:24.698283 23422 webserver.cc:460] Webserver started at http://127.17.95.193:42043/ using document root <none> and password file <none>
I20260927 11:08:24.698491 23422 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:24.698534 23422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:24.698684 23422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:24.699718 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "ab3ad20ad00b4f26970eca501a62a3b5"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "a1f75a636cd9a68c6513967f1b8fde4c"
server_key_iv: "08cb9531689496ab90311c8070b8aad2"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.700050 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "ab3ad20ad00b4f26970eca501a62a3b5"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "a1f75a636cd9a68c6513967f1b8fde4c"
server_key_iv: "08cb9531689496ab90311c8070b8aad2"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.701313 23422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:24.702003 23437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.702280 23422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:24.702353 23422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "ab3ad20ad00b4f26970eca501a62a3b5"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "a1f75a636cd9a68c6513967f1b8fde4c"
server_key_iv: "08cb9531689496ab90311c8070b8aad2"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.702431 23422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:24.711691 23422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:24.713451 23422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:24.713593 23422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:24.713788 23422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:24.714056 23422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:24.714087 23422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.714105 23422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:24.714145 23422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.719910 23422 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41977
I20260927 11:08:24.719959 23550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41977 every 8 connection(s)
I20260927 11:08:24.720260 23422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:24.728844 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:24.720911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:57253 (local address 127.17.95.254:40615)
0927 11:08:24.720969 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:24.720971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:24.721174 (+ 203us) server_negotiation.cc:407] Connection header received
0927 11:08:24.722009 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:24.722010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:24.722018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:24.722069 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:24.722608 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.723218 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.724169 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.724370 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.724425 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:24.726360 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:24.726361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:24.726362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:24.726368 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:24.727728 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:24.728073 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:24.728073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:24.728074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:24.728122 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:24.728379 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:24.728380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:24.728380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:24.728484 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:24.728536 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:24.728727 (+ 191us) server_negotiation.cc:299] Negotiation successful
0927 11:08:24.728785 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 11:08:24.729002 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:24.720900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40615 (local address 127.17.95.193:57253)
0927 11:08:24.721052 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:24.721092 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:24.721918 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:24.722117 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:24.722120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:24.722336 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:24.722511 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:24.722515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.723302 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.723303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:24.724092 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:24.724094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.724439 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.724439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:24.724548 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:24.724988 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:24.724998 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:24.726280 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:24.727799 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:24.727805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:24.727806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:24.728009 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:24.728196 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:24.728197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:24.728198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:24.728313 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:24.728538 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:24.728543 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:24.728639 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 11:08:24.728715 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 11:08:24.728821 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260927 11:08:24.729494 23551 heartbeater.cc:347] Connected to a master server at 127.17.95.254:40615
I20260927 11:08:24.729588 23551 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:24.729772 23551 heartbeater.cc:511] Master 127.17.95.254:40615 requested a full tablet report, sending...
I20260927 11:08:24.729764 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23422
I20260927 11:08:24.729864 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:24.730036 17791 external_mini_cluster.cc:1428] Setting key 8bdd704946f38ca64f39bc5531a5f466
I20260927 11:08:24.730315 23360 ts_manager.cc:194] Registered new tserver with Master: ab3ad20ad00b4f26970eca501a62a3b5 (127.17.95.193:41977)
I20260927 11:08:24.730957 23360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:57253
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:24.754207 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:40615
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:24.831745 23556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:24.831918 23556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:24.831938 23556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:24.833031 23556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:24.833072 23556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:24.833122 23556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:24.834522 23556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:40615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:24.834736 23556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:24.834940 23556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:24.840032 23562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:24.840029 23564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.840143 23556 server_base.cc:1034] running on GCE node
W20260927 11:08:24.840027 23561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.840430 23556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:24.840657 23556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:24.841809 23556 hybrid_clock.cc:648] HybridClock initialized: now 1790507304841804 us; error 33 us; skew 500 ppm
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:24.844221 23556 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:24.844612 23556 webserver.cc:460] Webserver started at http://127.17.95.194:38427/ using document root <none> and password file <none>
I20260927 11:08:24.844792 23556 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:24.844830 23556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:24.844910 23556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:24.845839 23556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "2f65fc5479ff4e0ab211148b4c9df0b1"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "9ae4895036221a0e9559bcee9d745933"
server_key_iv: "e6527009c674fe0cfced16c6ecb8908d"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.846143 23556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "2f65fc5479ff4e0ab211148b4c9df0b1"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "9ae4895036221a0e9559bcee9d745933"
server_key_iv: "e6527009c674fe0cfced16c6ecb8908d"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.847470 23556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:24.848120 23571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.848272 23556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:24.848371 23556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "2f65fc5479ff4e0ab211148b4c9df0b1"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "9ae4895036221a0e9559bcee9d745933"
server_key_iv: "e6527009c674fe0cfced16c6ecb8908d"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.848440 23556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:24.862253 23556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:24.863976 23556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:24.864164 23556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:24.864349 23556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:24.864596 23556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:24.864643 23556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.864678 23556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:24.864701 23556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.870196 23556 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:45257
I20260927 11:08:24.870263 23684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:45257 every 8 connection(s)
I20260927 11:08:24.870536 23556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:24.878489 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23556
I20260927 11:08:24.878607 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:24.878800 17791 external_mini_cluster.cc:1428] Setting key b0cea37a1c083024bf7396c4b75e7319
I20260927 11:08:24.879511 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:24.871260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:53997 (local address 127.17.95.254:40615)
0927 11:08:24.871301 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:24.871303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:24.871494 (+ 191us) server_negotiation.cc:407] Connection header received
0927 11:08:24.872347 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:24.872349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:24.872362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:24.872431 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:24.872996 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.873513 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.874407 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.874605 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.874668 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:24.876790 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:24.876792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:24.876792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:24.876799 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:24.878007 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:24.878439 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:24.878440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:24.878440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:24.878511 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:24.878779 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:24.878780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:24.878780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:24.878931 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:24.879028 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:24.879322 (+ 294us) server_negotiation.cc:299] Negotiation successful
0927 11:08:24.879409 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:24.879670 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:24.871238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40615 (local address 127.17.95.194:53997)
0927 11:08:24.871370 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:24.871408 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:24.872240 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:24.872482 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:24.872486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:24.872707 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:24.872903 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:24.872906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.873599 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.873600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:24.874338 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:24.874339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:24.874703 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:24.874704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:24.874817 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:24.875303 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:24.875311 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:24.876704 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:24.878089 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:24.878094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:24.878096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:24.878368 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:24.878598 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:24.878599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:24.878599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:24.878707 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:24.879018 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:24.879024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:24.879182 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 11:08:24.879293 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 11:08:24.879437 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260927 11:08:24.880134 23685 heartbeater.cc:347] Connected to a master server at 127.17.95.254:40615
I20260927 11:08:24.880232 23685 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:24.880402 23685 heartbeater.cc:511] Master 127.17.95.254:40615 requested a full tablet report, sending...
I20260927 11:08:24.880795 23360 ts_manager.cc:194] Registered new tserver with Master: 2f65fc5479ff4e0ab211148b4c9df0b1 (127.17.95.194:45257)
I20260927 11:08:24.881194 23360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:53997
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:24.902253 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:40615
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:24.982033 23690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:24.982256 23690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:24.982293 23690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:24.983435 23690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:24.983501 23690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:24.983546 23690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:24.984990 23690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:40615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:24.985239 23690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:24.985446 23690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:24.990247 23696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:24.990239 23695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.990271 23690 server_base.cc:1034] running on GCE node
W20260927 11:08:24.990237 23698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:24.990604 23690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:24.990828 23690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:24.991989 23690 hybrid_clock.cc:648] HybridClock initialized: now 1790507304991969 us; error 31 us; skew 500 ppm
Sep 27 11:08:24 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:24.994450 23690 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:24.994838 23690 webserver.cc:460] Webserver started at http://127.17.95.195:39859/ using document root <none> and password file <none>
I20260927 11:08:24.995060 23690 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:24.995157 23690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:24.995303 23690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:24.996341 23690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "e0817ced5df14ff78d39f15b0b95f968"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "d548cc40323210025a33b6c665d4528a"
server_key_iv: "3dfbc7d00f1c68d240c455f7935564ab"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.996680 23690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "e0817ced5df14ff78d39f15b0b95f968"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "d548cc40323210025a33b6c665d4528a"
server_key_iv: "3dfbc7d00f1c68d240c455f7935564ab"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.998031 23690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:24.998780 23705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:24.998943 23690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:24.999032 23690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "e0817ced5df14ff78d39f15b0b95f968"
format_stamp: "Formatted at 2026-09-27 11:08:24 on dist-test-slave-6j0v"
server_key: "d548cc40323210025a33b6c665d4528a"
server_key_iv: "3dfbc7d00f1c68d240c455f7935564ab"
server_key_version: "encryptionkey@0"
I20260927 11:08:24.999119 23690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:25.017930 23690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:25.020016 23690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:25.020170 23690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:25.020373 23690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:25.020656 23690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:25.020702 23690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:25.020740 23690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:25.020766 23690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:25.026178 23690 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:40931
I20260927 11:08:25.026261 23818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:40931 every 8 connection(s)
I20260927 11:08:25.026540 23690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:25.026669 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23690
I20260927 11:08:25.026791 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:25.027021 17791 external_mini_cluster.cc:1428] Setting key ff62e66a18183a2870199cec4ffe78a0
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507304, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:25.034916 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:25.027360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:37285 (local address 127.17.95.254:40615)
0927 11:08:25.027415 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.027418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.027591 (+ 173us) server_negotiation.cc:407] Connection header received
0927 11:08:25.028578 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.028579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.028588 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.028630 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.029161 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.029626 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.030493 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.030668 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.030715 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:25.032690 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:25.032691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:25.032691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:25.032698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:25.033821 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.034178 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.034179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.034179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.034224 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.034428 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.034429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.034429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.034522 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:25.034570 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:25.034794 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 11:08:25.034856 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 11:08:25.035102 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:25.027344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40615 (local address 127.17.95.195:37285)
0927 11:08:25.027486 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:25.027525 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:25.028495 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:25.028682 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:25.028686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:25.028918 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:25.029096 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:25.029099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.029695 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.029695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:25.030429 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:25.030431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.030729 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.030729 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:25.030815 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:25.031234 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:25.031242 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:25.032597 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:25.033885 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:25.033890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:25.033891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:25.034120 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:25.034275 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:25.034275 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:25.034275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:25.034380 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:25.034570 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:25.034574 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:25.034676 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 11:08:25.034753 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 11:08:25.034869 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260927 11:08:25.035542 23819 heartbeater.cc:347] Connected to a master server at 127.17.95.254:40615
I20260927 11:08:25.035612 23819 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:25.035734 23819 heartbeater.cc:511] Master 127.17.95.254:40615 requested a full tablet report, sending...
I20260927 11:08:25.036042 23360 ts_manager.cc:194] Registered new tserver with Master: e0817ced5df14ff78d39f15b0b95f968 (127.17.95.195:40931)
I20260927 11:08:25.036373 23360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:37285
I20260927 11:08:25.038682 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/test-admin.keytab.
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.053344 17791 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 11:08:25.054859 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:25.054123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.17.95.193:41977)
0927 11:08:25.054288 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.054292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.054302 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:25.054338 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.054340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.054353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.054415 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.054771 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37354: BlockingRecv error: recv got EOF from 127.0.0.1:37354 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
W20260927 11:08:25.054924 17791 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc)
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.055874 17791 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.056739 17791 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:25.063792 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:25.057232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37370 (local address 127.17.95.193:41977)
0927 11:08:25.057279 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.057281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.057321 (+ 40us) server_negotiation.cc:407] Connection header received
0927 11:08:25.057368 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.057369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.057377 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.057426 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.057763 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.058449 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.059160 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.059453 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.059544 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:25.061442 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:25.061449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:25.061449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:25.061455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:25.062806 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.063151 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.063153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.063153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.063202 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.063342 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.063342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.063343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.063446 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:25.063590 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:25.063628 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 11:08:25.063684 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 11:08:25.065428 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:25.064920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37378 (local address 127.17.95.193:41977)
0927 11:08:25.064980 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.064982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.064992 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:25.065050 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.065051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.065058 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.065091 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.065391 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37378: BlockingRecv error: recv got EOF from 127.0.0.1:37378 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260927 11:08:25.065485 17791 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc' of type FILE
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.066349 17791 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.067225 17791 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:25.073458 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:25.067702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37386 (local address 127.17.95.193:41977)
0927 11:08:25.067735 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.067736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.067765 (+ 29us) server_negotiation.cc:407] Connection header received
0927 11:08:25.067811 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.067811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.067817 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.067857 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.068183 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.068686 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.069330 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.069526 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.069583 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:25.071398 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:25.071399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:25.071399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:25.071405 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:25.072676 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.072966 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.072967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.072967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.073013 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.073149 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.073150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.073150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.073220 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:25.073281 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:25.073356 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 11:08:25.073408 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 11:08:25.074762 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:25.074354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.17.95.193:41977)
0927 11:08:25.074383 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.074384 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.074432 (+ 48us) server_negotiation.cc:407] Connection header received
0927 11:08:25.074475 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.074476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.074482 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.074524 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.074737 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37388: BlockingRecv error: recv got EOF from 127.0.0.1:37388 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260927 11:08:25.074906 17791 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790507277271108-17791-0/krb5kdc/krb5cc' of type FILE
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.075822 17791 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:25.076608 17791 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:08:25.082718 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:25.077047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37398 (local address 127.17.95.193:41977)
0927 11:08:25.077102 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:25.077104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:25.077168 (+ 64us) server_negotiation.cc:407] Connection header received
0927 11:08:25.077247 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:25.077248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:25.077256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:25.077316 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:25.077676 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.078121 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.078749 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:25.078925 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:25.079007 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:25.080855 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:25.080856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:25.080856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:25.080861 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:25.081992 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.082230 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.082231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.082231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.082271 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:25.082389 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:25.082390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:25.082390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:25.082493 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:25.082549 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:25.082626 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 11:08:25.082672 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 11:08:25.083238 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23422
I20260927 11:08:25.088581 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23556
I20260927 11:08:25.092715 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23690
I20260927 11:08:25.096613 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23328
2026-09-27T11:08:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1776 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23839](info): set up 2 sockets
Sep 27 11:08:25 dist-test-slave-6j0v krb5kdc[23839](info): commencing operation
krb5kdc: starting...
W20260927 11:08:26.131749 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab.
Entry for principal oryx/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab.
I20260927 11:08:26.242466 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34341
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab
--principal=oryx/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:26.319373 23852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:26.319578 23852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:26.319610 23852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:26.320657 23852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:26.320709 23852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:26.320732 23852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:26.320760 23852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:26.320786 23852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:26.322271 23852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab
--principal=oryx/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:26.322528 23852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:26.322765 23852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:26.327313 23860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.327430 23857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.327322 23858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:26.327322 23852 server_base.cc:1034] running on GCE node
I20260927 11:08:26.327723 23852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:26.327996 23852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:26.329150 23852 hybrid_clock.cc:648] HybridClock initialized: now 1790507306329140 us; error 36 us; skew 500 ppm
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:26.336093 23852 init.cc:377] Logged in from keytab as oryx/127.17.95.254@KRBTEST.COM (short username oryx)
I20260927 11:08:26.336596 23852 webserver.cc:460] Webserver started at http://127.17.95.254:39085/ using document root <none> and password file <none>
I20260927 11:08:26.336814 23852 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:26.336875 23852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:26.337013 23852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:26.338013 23852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "65f7cffdc5d24037a5c02798d6935c27"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "19704113898c5823575125b992bce04f"
server_key_iv: "9c6964b90e0949928994380a5a89b475"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.338348 23852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "65f7cffdc5d24037a5c02798d6935c27"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "19704113898c5823575125b992bce04f"
server_key_iv: "9c6964b90e0949928994380a5a89b475"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.339640 23852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:26.340252 23867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.340394 23852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:26.340523 23852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "65f7cffdc5d24037a5c02798d6935c27"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "19704113898c5823575125b992bce04f"
server_key_iv: "9c6964b90e0949928994380a5a89b475"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.340588 23852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:26.361826 23852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:26.364017 23852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:26.364176 23852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:26.368402 23852 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:34341
I20260927 11:08:26.368441 23919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:34341 every 8 connection(s)
I20260927 11:08:26.368808 23852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:26.369788 23920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:26.372191 23920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Bootstrap starting.
I20260927 11:08:26.372763 23920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:26.373052 23920 log.cc:824] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:26.373751 23920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: No bootstrap required, opened a new log
I20260927 11:08:26.375356 23920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } }
I20260927 11:08:26.375487 23920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:26.375509 23920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f7cffdc5d24037a5c02798d6935c27, State: Initialized, Role: FOLLOWER
I20260927 11:08:26.375667 23920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } }
I20260927 11:08:26.375726 23920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:26.375763 23920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:26.375808 23920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:26.376602 23920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } }
I20260927 11:08:26.376703 23920 leader_election.cc:302] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f7cffdc5d24037a5c02798d6935c27; no voters:
I20260927 11:08:26.376853 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23852
I20260927 11:08:26.376891 23920 leader_election.cc:288] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:26.376911 23923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:26.376969 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:26.377054 23923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 LEADER]: Becoming Leader. State: Replica: 65f7cffdc5d24037a5c02798d6935c27, State: Running, Role: LEADER
I20260927 11:08:26.377125 23923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } }
I20260927 11:08:26.377151 17791 external_mini_cluster.cc:1428] Setting key 335a6b39a3a672097d7b0f93b896ca65
I20260927 11:08:26.377148 23920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:26.378049 23925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f7cffdc5d24037a5c02798d6935c27. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } }
I20260927 11:08:26.378136 23925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:26.378206 23924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } }
I20260927 11:08:26.378263 23924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:26.378458 23935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:26.379050 23935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:26.380522 23935 catalog_manager.cc:1347] Generated new cluster ID: a00062d2db18484596fba69a1bddb89b
I20260927 11:08:26.380585 23935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:26.388912 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.377782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57630 (local address 127.17.95.254:34341)
0927 11:08:26.378041 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:26.378051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:26.378087 (+ 36us) server_negotiation.cc:407] Connection header received
0927 11:08:26.378228 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:26.378235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:26.378292 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:26.379501 (+ 1209us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:26.379591 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.380882 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.381905 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.382515 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.382621 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.385253 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:26.385257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:26.385257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:26.385269 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:26.387279 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.387639 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.387642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.387642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.387762 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.387944 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.387945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.387946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.388260 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:26.388424 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:26.388532 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 11:08:26.388696 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":77,"threads_started":1}
I20260927 11:08:26.400382 23935 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:26.400938 23935 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:26.404898 23935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Generated new TSK 0
I20260927 11:08:26.405071 23935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab.
Entry for principal oryx/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab.
I20260927 11:08:26.430151 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34341
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:26.514056 23946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:26.514272 23946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:26.514312 23946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:26.515501 23946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:26.515560 23946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:26.515609 23946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:26.517062 23946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:26.517308 23946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:26.517547 23946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:26.522383 23954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.522394 23951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.522446 23952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:26.522619 23946 server_base.cc:1034] running on GCE node
I20260927 11:08:26.522888 23946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:26.523164 23946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:26.524359 23946 hybrid_clock.cc:648] HybridClock initialized: now 1790507306524337 us; error 27 us; skew 500 ppm
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:26.526763 23946 init.cc:377] Logged in from keytab as oryx/127.17.95.193@KRBTEST.COM (short username oryx)
I20260927 11:08:26.527211 23946 webserver.cc:460] Webserver started at http://127.17.95.193:46005/ using document root <none> and password file <none>
I20260927 11:08:26.527535 23946 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:26.527602 23946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:26.527729 23946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:26.528723 23946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "3ad61acf3e76459d93d1f114fdd47ab9"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "3314e727c75c51446ec123f9025dc6e7"
server_key_iv: "b9d32ada144a78a6df1903457ca5fea8"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.529072 23946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "3ad61acf3e76459d93d1f114fdd47ab9"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "3314e727c75c51446ec123f9025dc6e7"
server_key_iv: "b9d32ada144a78a6df1903457ca5fea8"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.530313 23946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:26.531049 23961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.531279 23946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:08:26.531366 23946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "3ad61acf3e76459d93d1f114fdd47ab9"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "3314e727c75c51446ec123f9025dc6e7"
server_key_iv: "b9d32ada144a78a6df1903457ca5fea8"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.531443 23946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:26.574158 23946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:26.576195 23946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:26.576352 23946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:26.576586 23946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:26.576941 23946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:26.576993 23946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.577035 23946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:26.577064 23946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.582594 23946 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:45643
I20260927 11:08:26.582665 24074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:45643 every 8 connection(s)
I20260927 11:08:26.582969 23946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:26.585268 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 23946
I20260927 11:08:26.585356 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:26.585541 17791 external_mini_cluster.cc:1428] Setting key 193ecd0ded767b6e44eb09d32877eccd
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:26.593799 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.583648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:34441 (local address 127.17.95.254:34341)
0927 11:08:26.583698 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:26.583701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:26.583908 (+ 207us) server_negotiation.cc:407] Connection header received
0927 11:08:26.584398 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:26.584399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:26.584408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:26.584482 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:26.585470 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.586123 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.587801 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.588133 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.588210 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.590547 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:26.590549 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:26.590549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:26.590558 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:26.592419 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.592813 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.592814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.592814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.592882 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.593117 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.593119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.593119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.593260 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:26.593325 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:26.593614 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 11:08:26.593698 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:26.593832 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.583638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:34441)
0927 11:08:26.583780 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:26.583812 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:26.584314 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:26.584522 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:26.584525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:26.585097 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:26.585382 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:26.585389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.586238 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.586239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:26.587705 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:26.587709 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.588223 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.588224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:26.588327 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.588961 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:26.588985 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:26.590428 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:26.592512 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:26.592518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:26.592520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:26.592737 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:26.592956 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:26.592957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:26.592957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:26.593053 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:26.593332 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:26.593336 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:26.593458 (+ 122us) client_negotiation.cc:769] Sending connection context
0927 11:08:26.593560 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 11:08:26.593678 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260927 11:08:26.594378 24075 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34341
I20260927 11:08:26.594511 24075 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.17.95.194@KRBTEST.COM; defaulting to no policy
I20260927 11:08:26.594708 24075 heartbeater.cc:511] Master 127.17.95.254:34341 requested a full tablet report, sending...
I20260927 11:08:26.595263 23884 ts_manager.cc:194] Registered new tserver with Master: 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643)
I20260927 11:08:26.595933 23884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:34441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab.
Entry for principal oryx/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab.
I20260927 11:08:26.611811 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34341
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:26.689502 24080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:26.689677 24080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:26.689698 24080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:26.690802 24080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:26.690841 24080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:26.690865 24080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:26.692281 24080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:26.692534 24080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:26.692773 24080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:26.697558 24086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.697558 24085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:26.697683 24080 server_base.cc:1034] running on GCE node
W20260927 11:08:26.697733 24088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:26.697983 24080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:26.698181 24080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:26.699334 24080 hybrid_clock.cc:648] HybridClock initialized: now 1790507306699317 us; error 28 us; skew 500 ppm
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:26.701719 24080 init.cc:377] Logged in from keytab as oryx/127.17.95.194@KRBTEST.COM (short username oryx)
I20260927 11:08:26.702113 24080 webserver.cc:460] Webserver started at http://127.17.95.194:38777/ using document root <none> and password file <none>
I20260927 11:08:26.702306 24080 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:26.702346 24080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:26.702498 24080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:26.703504 24080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "44404d6e189f4c81a451dae98db9dac7"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "4674ed1c450fb01011983d9016ae65d7"
server_key_iv: "c218933e07f681a84b7e7e57fe3c3abd"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.703827 24080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "44404d6e189f4c81a451dae98db9dac7"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "4674ed1c450fb01011983d9016ae65d7"
server_key_iv: "c218933e07f681a84b7e7e57fe3c3abd"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.705101 24080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:26.705848 24095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.706040 24080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:26.706156 24080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "44404d6e189f4c81a451dae98db9dac7"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "4674ed1c450fb01011983d9016ae65d7"
server_key_iv: "c218933e07f681a84b7e7e57fe3c3abd"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.706221 24080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:26.723902 24080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:26.725708 24080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:26.725850 24080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:26.726047 24080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:26.726328 24080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:26.726378 24080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.726416 24080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:26.726442 24080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.731982 24080 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:34313
I20260927 11:08:26.732025 24208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:34313 every 8 connection(s)
I20260927 11:08:26.732312 24080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:26.736092 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24080
I20260927 11:08:26.736174 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:26.736387 17791 external_mini_cluster.cc:1428] Setting key 6c5ec7366f259a3a3bb217ba3c844ffd
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:26.741760 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.732964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:47669)
0927 11:08:26.733145 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:26.733179 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:26.733715 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:26.733993 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:26.733996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:26.734541 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:26.734715 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:26.734720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.735470 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.735471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:26.736369 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:26.736372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.736827 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.736828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:26.736952 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.737471 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:26.737478 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:26.738744 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:26.740460 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:26.740465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:26.740466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:26.740669 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:26.740890 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:26.740891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:26.740891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:26.741028 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:26.741316 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:26.741320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:26.741422 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 11:08:26.741501 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:08:26.741614 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260927 11:08:26.741772 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.733011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:47669 (local address 127.17.95.254:34341)
0927 11:08:26.733079 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:26.733082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:26.733267 (+ 185us) server_negotiation.cc:407] Connection header received
0927 11:08:26.733870 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:26.733871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:26.733882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:26.733936 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:26.734809 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.735389 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.736465 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.736752 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.736822 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.738840 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:26.738842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:26.738843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:26.738850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:26.740387 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.740738 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.740739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.740740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.740801 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.741102 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.741103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.741103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.741245 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:26.741313 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:26.741548 (+ 235us) server_negotiation.cc:299] Negotiation successful
0927 11:08:26.741636 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:08:26.742300 24209 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34341
I20260927 11:08:26.742394 24209 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:26.742563 24209 heartbeater.cc:511] Master 127.17.95.254:34341 requested a full tablet report, sending...
I20260927 11:08:26.742951 23884 ts_manager.cc:194] Registered new tserver with Master: 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313)
I20260927 11:08:26.743494 23884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:47669
WARNING: no policy specified for oryx/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab.
Entry for principal oryx/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab.
I20260927 11:08:26.759871 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34341
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:26.836642 24214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:26.836823 24214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:26.836845 24214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:26.837963 24214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:26.838002 24214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:26.838027 24214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:26.839411 24214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:34341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:26.839638 24214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:26.839860 24214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:26.844465 24222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.844472 24219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:26.844481 24220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:26.844688 24214 server_base.cc:1034] running on GCE node
I20260927 11:08:26.844864 24214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:26.845149 24214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:26.846309 24214 hybrid_clock.cc:648] HybridClock initialized: now 1790507306846309 us; error 41 us; skew 500 ppm
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:26.848716 24214 init.cc:377] Logged in from keytab as oryx/127.17.95.195@KRBTEST.COM (short username oryx)
I20260927 11:08:26.849062 24214 webserver.cc:460] Webserver started at http://127.17.95.195:44547/ using document root <none> and password file <none>
I20260927 11:08:26.849236 24214 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:26.849272 24214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:26.849352 24214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:26.850255 24214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "facf2d136f2549d1872c3ec4d5705f53"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "56c92054dd75d07330ab6d6adeb967d2"
server_key_iv: "4f354da7eda3706693e81efbec0b8a38"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.850538 24214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "facf2d136f2549d1872c3ec4d5705f53"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "56c92054dd75d07330ab6d6adeb967d2"
server_key_iv: "4f354da7eda3706693e81efbec0b8a38"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.851853 24214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:26.852459 24229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.852622 24214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:26.852703 24214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "facf2d136f2549d1872c3ec4d5705f53"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "56c92054dd75d07330ab6d6adeb967d2"
server_key_iv: "4f354da7eda3706693e81efbec0b8a38"
server_key_version: "encryptionkey@0"
I20260927 11:08:26.852753 24214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:26.883167 24214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:26.885262 24214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:26.885362 24214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:26.885527 24214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:26.885818 24214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:26.885850 24214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.885890 24214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:26.885915 24214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:26.891640 24214 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:45481
I20260927 11:08:26.891723 24342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:45481 every 8 connection(s)
I20260927 11:08:26.892016 24214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:26.893999 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24214
I20260927 11:08:26.894093 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:26.894261 17791 external_mini_cluster.cc:1428] Setting key 7ce30a7ef75ffa591a814740f4934df8
Sep 27 11:08:26 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:26.901471 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.892688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:59609 (local address 127.17.95.254:34341)
0927 11:08:26.892736 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:26.892738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:26.892938 (+ 200us) server_negotiation.cc:407] Connection header received
0927 11:08:26.893434 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:26.893435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:26.893444 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:26.893495 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:26.894498 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.895300 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.896894 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.897090 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.897148 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.898996 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:26.898997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:26.898998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:26.899004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:26.900285 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.900604 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.900605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.900605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.900649 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:26.900925 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:26.900926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:26.900927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:26.901053 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:26.901121 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:26.901339 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 11:08:26.901403 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:26.901638 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:26.892679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.195:59609)
0927 11:08:26.892827 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:26.892860 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:26.893347 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:26.893545 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:26.893548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:26.894133 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:26.894372 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:26.894378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.895423 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.895424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:26.896817 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:26.896820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:26.897158 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:26.897159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:26.897255 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:26.897641 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:26.897648 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:26.898922 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:26.900344 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:26.900348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:26.900349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:26.900548 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:26.900697 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:26.900697 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:26.900697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:26.900798 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:26.901121 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:26.901125 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:26.901227 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 11:08:26.901309 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:08:26.901411 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260927 11:08:26.902145 24343 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34341
I20260927 11:08:26.902233 24343 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:26.902391 24343 heartbeater.cc:511] Master 127.17.95.254:34341 requested a full tablet report, sending...
I20260927 11:08:26.902767 23884 ts_manager.cc:194] Registered new tserver with Master: facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481)
I20260927 11:08:26.903183 23884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:59609
I20260927 11:08:26.905934 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.95.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.253.keytab.
Entry for principal oryx/127.17.95.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.253.keytab.
I20260927 11:08:26.929894 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:36361
--webserver_interface=127.17.95.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.253.keytab
--principal=oryx/127.17.95.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34341,127.17.95.253:36361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:27.008230 24348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:27.008414 24348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:27.008435 24348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:27.009593 24348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:27.009631 24348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:27.009645 24348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:27.009657 24348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:27.009698 24348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:27.011004 24348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34341,127.17.95.253:36361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.253.keytab
--principal=oryx/127.17.95.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.95.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
W20260927 11:08:27.011133 24348 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.95.254:34341,127.17.95.253:36361'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 11:08:27.011283 24348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:27.011484 24348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.015985 24353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.016021 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.016103 24348 server_base.cc:1034] running on GCE node
W20260927 11:08:27.016034 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.016363 24348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.016664 24348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.017823 24348 hybrid_clock.cc:648] HybridClock initialized: now 1790507307017810 us; error 25 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.020193 24348 init.cc:377] Logged in from keytab as oryx/127.17.95.253@KRBTEST.COM (short username oryx)
I20260927 11:08:27.020603 24348 webserver.cc:460] Webserver started at http://127.17.95.253:39741/ using document root <none> and password file <none>
I20260927 11:08:27.020808 24348 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.020869 24348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.021003 24348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:27.021937 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/instance:
uuid: "d8d0490bd71446f48b3fee40f9f70383"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "ef0fd875e6c97416ac755461dea4332f"
server_key_iv: "1199f1400eb5149fd10525728cee2129"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.022262 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal/instance:
uuid: "d8d0490bd71446f48b3fee40f9f70383"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "ef0fd875e6c97416ac755461dea4332f"
server_key_iv: "1199f1400eb5149fd10525728cee2129"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.023581 24348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:27.024242 24363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.024394 24348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:27.024492 24348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
uuid: "d8d0490bd71446f48b3fee40f9f70383"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "ef0fd875e6c97416ac755461dea4332f"
server_key_iv: "1199f1400eb5149fd10525728cee2129"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.024555 24348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.032436 24348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.034264 24348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:27.034397 24348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.038204 24348 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.253:36361
I20260927 11:08:27.038234 24415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.253:36361 every 8 connection(s)
I20260927 11:08:27.038542 24348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
I20260927 11:08:27.039547 24416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:27.040896 24416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:08:27.044107 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24348
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.253@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
I20260927 11:08:27.049916 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.041559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57638 (local address 127.17.95.254:34341)
0927 11:08:27.041615 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.041617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.041786 (+ 169us) server_negotiation.cc:407] Connection header received
0927 11:08:27.042181 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.042182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.042191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.042247 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.042811 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.043466 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.044667 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.044871 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.044956 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.046967 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.046968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.046969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.046984 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.048502 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.048896 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.048897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.048897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.048956 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.049199 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.049200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.049200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.049395 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.049468 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.049732 (+ 264us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.049811 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 11:08:27.049988 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.041515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:57638)
0927 11:08:27.041663 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.041701 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.042107 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.042308 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.042311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.042564 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.042733 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.042736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.043530 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.043530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.044587 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.044589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.044965 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.044966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.045089 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.045593 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.045600 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.046871 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.048597 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.048603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.048605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.048812 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.049041 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.049042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.049042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.049136 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.049475 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.049479 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.049583 (+ 104us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.049714 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.049823 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260927 11:08:27.050559 24416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:08:27.053303 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.044587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37610 (local address 127.17.95.253:36361)
0927 11:08:27.044898 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.044901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.044914 (+ 13us) server_negotiation.cc:407] Connection header received
0927 11:08:27.044930 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.044932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.044945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.045024 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.045753 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.046403 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.047292 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.047568 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.047651 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.050389 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.050395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.050395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.050401 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.052183 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.052498 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.052501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.052501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.052560 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.052725 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.052726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.052726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.052875 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.053004 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.053096 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.053188 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":91,"threads_started":1}
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.253@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
I20260927 11:08:27.057781 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.050794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37614)
0927 11:08:27.050855 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.050866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.050906 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.051085 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.051086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.051270 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.051325 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.051327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.052047 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.052048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.052744 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.052746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.053142 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.053143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.053211 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.053532 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.053536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.054691 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.056738 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.056739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.056740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.056904 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.057120 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.057120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.057121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.057176 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.057439 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.057439 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.057487 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.057580 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.057673 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 11:08:27.057876 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.050832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.17.95.253:36361)
0927 11:08:27.050959 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.050960 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.050966 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:08:27.050981 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.050981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.050989 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.051042 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.051396 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.051957 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.052816 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.053080 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.053136 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.054776 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.054777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.054777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.054783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.056641 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.056979 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.056981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.056981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.057037 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.057255 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.057256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.057256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.057362 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.057446 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.057605 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.057725 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260927 11:08:27.058988 24416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Bootstrap starting.
I20260927 11:08:27.059619 24416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:27.059897 24416 log.cc:824] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:27.060561 24416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: No bootstrap required, opened a new log
I20260927 11:08:27.061903 24416 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } }
I20260927 11:08:27.062027 24416 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:27.062052 24416 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8d0490bd71446f48b3fee40f9f70383, State: Initialized, Role: FOLLOWER
I20260927 11:08:27.062199 24416 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } }
I20260927 11:08:27.062439 24423 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } } }
I20260927 11:08:27.062494 24416 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:27.062527 24423 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 11:08:27.064170 24434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:08:27.064222 24434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:08:27.065716 24348 master_runner.cc:428] Detected that this master d8d0490bd71446f48b3fee40f9f70383 is joining an existing cluster
I20260927 11:08:27.065785 24348 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.95.253:36361
I20260927 11:08:27.073585 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.066781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.17.95.254:34341)
0927 11:08:27.066825 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.066827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.066959 (+ 132us) server_negotiation.cc:407] Connection header received
0927 11:08:27.067023 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.067024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.067034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.067120 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.067526 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.068253 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.069215 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.069414 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.069520 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.070567 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.070568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.070568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.070575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.072292 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.072761 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.072763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.072763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.072823 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.073064 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.073065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.073065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.073195 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.073270 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.073410 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.073483 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:27.073659 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.066914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.17.95.253:36361)
0927 11:08:27.066957 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.066959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.067152 (+ 193us) server_negotiation.cc:407] Connection header received
0927 11:08:27.067240 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.067241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.067249 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.067302 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.067660 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.068111 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.069151 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.069451 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.069528 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.070744 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.070745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.070745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.070752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.072499 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.072867 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.072868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.072869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.072919 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.073121 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.073122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.073122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.073237 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.073298 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.073448 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.073550 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":18}
I20260927 11:08:27.073832 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.066741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:57644)
0927 11:08:27.066881 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.066892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.066957 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.067129 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.067130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.067360 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.067445 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.067447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.068338 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.068339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.069115 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.069117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.069537 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.069538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.069597 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.070043 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.070047 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.070461 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.072384 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.072385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.072385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.072677 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.072903 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.072904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.072904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.072985 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.073260 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.073261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.073302 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.073384 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.073628 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260927 11:08:27.073846 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.066782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37622)
0927 11:08:27.067058 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.067066 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.067171 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.067306 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.067306 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.067516 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.067574 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.067575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.068200 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.068201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.069065 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.069067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.069570 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.069570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.069665 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.070231 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.070236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.070661 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.072583 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.072584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.072584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.072800 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.072980 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.072980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.072981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.073055 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.073320 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.073321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.073352 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.073470 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.073612 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":47,"thread_start_us":62,"threads_started":1}
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.254@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
I20260927 11:08:27.083475 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.076267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37634)
0927 11:08:27.076462 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.076488 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.076548 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.076685 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.076687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.076913 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.076962 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.076963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.077863 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.077864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.078587 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.078590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.079033 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.079034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.079180 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.079591 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.079595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.080971 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.082479 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.082502 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.082504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.082672 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.082851 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.082851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.082851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.082899 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.083179 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.083185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.083230 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.083306 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.083403 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260927 11:08:27.083503 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.076335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.17.95.253:36361)
0927 11:08:27.076371 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.076372 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.076544 (+ 172us) server_negotiation.cc:407] Connection header received
0927 11:08:27.076607 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.076608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.076617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.076681 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.077015 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.077757 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.078659 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.078945 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.079013 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.081047 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.081049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.081049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.081056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.082397 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.082735 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.082736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.082737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.082788 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.082956 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.082957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.082957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.083114 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.083182 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.083352 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.083418 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:27.084362 23884 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.95.253:36361: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383"
member_type: NON_VOTER
last_known_addr {
host: "127.17.95.253"
port: 36361
}
attrs {
promote: true
}
}
dest_uuid: "65f7cffdc5d24037a5c02798d6935c27"
cas_config_opid_index: -1
I20260927 11:08:27.084653 23884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } }
I20260927 11:08:27.085103 24443 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d8d0490bd71446f48b3fee40f9f70383 (127.17.95.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.085407 24390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:27.086370 24443 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.95.253:36361
I20260927 11:08:27.086371 24390 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Refusing update from remote peer 65f7cffdc5d24037a5c02798d6935c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 11:08:27.086620 24348 master.cc:537] Master@127.17.95.253:36361 shutting down...
I20260927 11:08:27.086843 24444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.086843 24443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:27.086993 24444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
W20260927 11:08:27.087704 23870 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260927 11:08:27.089080 24348 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:27.089166 24348 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:27.089214 24348 tablet_replica.cc:330] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: stopping tablet replica
I20260927 11:08:27.099395 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.099280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37642)
0927 11:08:27.099332 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.099373 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:36361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 11:08:27.099498 23870 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 -> Peer d8d0490bd71446f48b3fee40f9f70383 (127.17.95.253:36361): Couldn't send request to peer d8d0490bd71446f48b3fee40f9f70383. Status: Network error: Client connection negotiation failed: client connection to 127.17.95.253:36361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 11:08:27.101569 24348 master.cc:552] Master@127.17.95.253:36361 shutdown complete.
I20260927 11:08:27.101613 24348 master_runner.cc:305] Clearing existing system tablet
I20260927 11:08:27.101737 24348 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.101780 24348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.102447 24348 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:27.102965 24449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.103130 24348 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.103184 24348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.103207 24348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
uuid: "d8d0490bd71446f48b3fee40f9f70383"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "ef0fd875e6c97416ac755461dea4332f"
server_key_iv: "1199f1400eb5149fd10525728cee2129"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.103847 24348 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:27.105326 24348 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 11:08:27.105397 24348 log.cc:1196] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 11:08:27.105594 24348 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Deleting consensus metadata
I20260927 11:08:27.105701 24348 master_runner.cc:315] Copying system tablet from 127.17.95.254:34341
I20260927 11:08:27.106222 24348 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: tablet copy: Beginning tablet copy session from remote peer at address 127.17.95.254:34341
I20260927 11:08:27.112138 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.106593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.17.95.254:34341)
0927 11:08:27.106634 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.106636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.106705 (+ 69us) server_negotiation.cc:407] Connection header received
0927 11:08:27.106776 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.106777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.106784 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.106824 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.107253 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.107758 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.108752 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.108921 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.108982 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.109886 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.109887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.109887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.109893 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.111250 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.111576 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.111577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.111577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.111619 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.111768 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.111768 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.111768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.111844 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.111883 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.112036 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.112089 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:08:27.112214 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.106513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:57658)
0927 11:08:27.106635 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.106649 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.106720 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.106848 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.106849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.107016 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.107163 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.107165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.107824 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.107825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.108680 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.108683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.108991 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.108992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.109054 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.109462 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.109466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.109813 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.111310 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.111311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.111311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.111515 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.111668 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.111668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.111669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.111719 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.111890 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.111891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.111928 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.112019 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.112106 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260927 11:08:27.112459 23904 tablet_copy_service.cc:140] P 65f7cffdc5d24037a5c02798d6935c27: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d8d0490bd71446f48b3fee40f9f70383 ({username='oryx', principal='oryx/127.17.95.253@KRBTEST.COM'} at 127.0.0.1:57658)
I20260927 11:08:27.112542 23904 tablet_copy_service.cc:161] P 65f7cffdc5d24037a5c02798d6935c27: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d8d0490bd71446f48b3fee40f9f70383 at {username='oryx', principal='oryx/127.17.95.253@KRBTEST.COM'} at 127.0.0.1:57658: session id = d8d0490bd71446f48b3fee40f9f70383-00000000000000000000000000000000
I20260927 11:08:27.113158 23904 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 11:08:27.113703 24348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 11:08:27.114440 24348 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: tablet copy: Starting download of 0 data blocks...
I20260927 11:08:27.114574 24348 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: tablet copy: Starting download of 1 WAL segments...
I20260927 11:08:27.115271 24348 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 11:08:27.116011 23904 tablet_copy_service.cc:342] P 65f7cffdc5d24037a5c02798d6935c27: Request end of tablet copy session d8d0490bd71446f48b3fee40f9f70383-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.95.253@KRBTEST.COM'} at 127.0.0.1:57658
I20260927 11:08:27.116060 23904 tablet_copy_service.cc:433] P 65f7cffdc5d24037a5c02798d6935c27: ending tablet copy session d8d0490bd71446f48b3fee40f9f70383-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d8d0490bd71446f48b3fee40f9f70383
I20260927 11:08:27.117933 24348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.118976 24457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.119159 24459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.119187 24348 server_base.cc:1034] running on GCE node
W20260927 11:08:27.119015 24456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.119414 24348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.119546 24348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.120672 24348 hybrid_clock.cc:648] HybridClock initialized: now 1790507307120661 us; error 24 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.122087 24348 init.cc:377] Logged in from keytab as oryx/127.17.95.253@KRBTEST.COM (short username oryx)
I20260927 11:08:27.122321 24348 webserver.cc:460] Webserver started at http://127.17.95.253:35771/ using document root <none> and password file <none>
I20260927 11:08:27.122438 24348 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.122486 24348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.123057 24348 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.123564 24466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.123701 24348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.123754 24348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
uuid: "d8d0490bd71446f48b3fee40f9f70383"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "ef0fd875e6c97416ac755461dea4332f"
server_key_iv: "1199f1400eb5149fd10525728cee2129"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.123801 24348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.131551 24348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.133102 24348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.135926 24348 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.253:36361
I20260927 11:08:27.136286 24518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.253:36361 every 8 connection(s)
I20260927 11:08:27.136289 24348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
I20260927 11:08:27.137025 24519 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:08:27.137425 24519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Bootstrap starting.
I20260927 11:08:27.139765 24519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:08:27.139981 24519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Bootstrap complete.
I20260927 11:08:27.140175 24519 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } }
I20260927 11:08:27.140242 24519 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d8d0490bd71446f48b3fee40f9f70383, State: Initialized, Role: LEARNER
I20260927 11:08:27.140287 24519 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } }
I20260927 11:08:27.140465 24519 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:27.140440 24522 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.140511 24522 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: LEARNER
I20260927 11:08:27.141788 24533 catalog_manager.cc:1259] Loaded cluster ID: a00062d2db18484596fba69a1bddb89b
I20260927 11:08:27.141831 24533 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: loading cluster ID for follower catalog manager: success
I20260927 11:08:27.142894 24533 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: acquiring CA information for follower catalog manager: success
I20260927 11:08:27.143240 24533 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:08:27.161774 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.154756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.17.95.253:36361)
0927 11:08:27.154847 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.154848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.154858 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:27.154886 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.154886 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.154895 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.154969 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.155758 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.156500 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.157557 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.157751 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.157809 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.159215 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.159217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.159217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.159223 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.160703 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.161080 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.161081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.161081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.161133 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.161329 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.161330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.161330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.161420 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.161499 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.161642 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.161701 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":56,"threads_started":1}
WARNING: no policy specified for oryx/127.17.95.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.252.keytab.
Entry for principal oryx/127.17.95.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.252.keytab.
I20260927 11:08:27.186923 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:33083
--webserver_interface=127.17.95.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.252.keytab
--principal=oryx/127.17.95.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:27.267488 24537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:27.267702 24537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:27.267742 24537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:27.268862 24537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:27.268925 24537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:27.268949 24537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:27.268990 24537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:27.269022 24537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:27.270431 24537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.252.keytab
--principal=oryx/127.17.95.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:33083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.95.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:27.270689 24537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:27.270924 24537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.275635 24545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.275648 24543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.275635 24542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.275652 24537 server_base.cc:1034] running on GCE node
I20260927 11:08:27.276054 24537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.276335 24537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.277482 24537 hybrid_clock.cc:648] HybridClock initialized: now 1790507307277471 us; error 50 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.280011 24537 init.cc:377] Logged in from keytab as oryx/127.17.95.252@KRBTEST.COM (short username oryx)
I20260927 11:08:27.280396 24537 webserver.cc:460] Webserver started at http://127.17.95.252:40875/ using document root <none> and password file <none>
I20260927 11:08:27.280610 24537 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.280670 24537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.280804 24537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:27.281774 24537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/instance:
uuid: "f1cba763d804494a926882ce62290ee2"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "22f932898810f737cd42103e28712713"
server_key_iv: "6357caba5e594399d2e7e52071391150"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.282085 24537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal/instance:
uuid: "f1cba763d804494a926882ce62290ee2"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "22f932898810f737cd42103e28712713"
server_key_iv: "6357caba5e594399d2e7e52071391150"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.283319 24537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:27.283973 24552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.284122 24537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:27.284207 24537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
uuid: "f1cba763d804494a926882ce62290ee2"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "22f932898810f737cd42103e28712713"
server_key_iv: "6357caba5e594399d2e7e52071391150"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.284274 24537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.297715 24537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.299448 24537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:27.299589 24537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.303447 24537 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.252:33083
I20260927 11:08:27.303624 24604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.252:33083 every 8 connection(s)
I20260927 11:08:27.304001 24537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
I20260927 11:08:27.304960 24605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:27.306357 24605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:08:27.311292 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24537
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.252@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:27.315107 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.307132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.17.95.254:34341)
0927 11:08:27.307188 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.307190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.307275 (+ 85us) server_negotiation.cc:407] Connection header received
0927 11:08:27.307774 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.307775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.307785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.307842 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.308394 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.309010 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.310011 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.310230 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.310305 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.312473 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.312474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.312475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.312481 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.313825 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.314203 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.314204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.314205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.314258 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.314475 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.314476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.314476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.314582 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.314652 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.314906 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.314993 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:08:27.315225 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.306973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:57666)
0927 11:08:27.307143 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.307182 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.307677 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.307906 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.307909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.308140 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.308313 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.308317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.309080 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.309081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.309934 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.309936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.310342 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.310343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.310463 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.310873 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.310880 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.312382 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.313914 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.313920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.313922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.314120 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.314316 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.314317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.314317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.314416 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.314642 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.314647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.314771 (+ 124us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.314867 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.315006 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":124,"threads_started":1}
I20260927 11:08:27.315779 24605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.95.252@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.252@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
I20260927 11:08:27.321102 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.311810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.17.95.252:33083)
0927 11:08:27.311943 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.311947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.312263 (+ 316us) server_negotiation.cc:407] Connection header received
0927 11:08:27.312320 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.312323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.312347 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.312417 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.313141 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.313888 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.314592 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.314865 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.314985 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.317802 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.317807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.317807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.317812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.319542 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.319896 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.319899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.319899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.319949 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.320158 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.320159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.320159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.320271 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.320410 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.320498 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.320572 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260927 11:08:27.322654 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.316007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37660)
0927 11:08:27.316039 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.316049 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.316089 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.316257 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.316261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.316359 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.316397 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.316398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.317150 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.317151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.317933 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.317935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.318356 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.318357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.318406 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.318809 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.318813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.320058 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.321918 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.321918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.321919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.322068 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.322219 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.322220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.322220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.322265 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.322423 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.322424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.322464 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.322526 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.322584 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 11:08:27.322736 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.316078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37660 (local address 127.17.95.253:36361)
0927 11:08:27.316132 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.316134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.316144 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:27.316175 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.316176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.316189 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.316257 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.316459 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.317061 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.318007 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.318286 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.318359 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.320127 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.320128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.320129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.320136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.321860 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.322126 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.322127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.322127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.322171 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.322300 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.322300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.322301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.322382 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.322424 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.322560 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.322655 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 11:08:27.323060 24605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.252@KRBTEST.COM for oryx/127.17.95.252@KRBTEST.COM
I20260927 11:08:27.328780 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.323311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:56342)
0927 11:08:27.323355 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.323365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.323402 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.323542 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.323543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.323664 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.323684 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.323686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.324271 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.324272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.324796 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.324797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.325154 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.325155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.325200 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.325503 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.325507 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.326511 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.327867 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.327867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.327868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.328023 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.328207 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.328208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.328209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.328269 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.328490 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.328491 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.328525 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.328588 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.328674 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 11:08:27.328830 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.323362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.17.95.252:33083)
0927 11:08:27.323399 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.323401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.323407 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:08:27.323452 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.323453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.323459 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.323540 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.323757 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.324200 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.324878 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.325081 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.325155 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.326600 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.326602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.326602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.326608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.327780 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.328099 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.328100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.328101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.328149 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.328340 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.328340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.328341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.328430 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.328496 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.328612 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.328720 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":19}
I20260927 11:08:27.330049 24605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Bootstrap starting.
I20260927 11:08:27.330847 24605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:27.331183 24605 log.cc:824] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:27.331846 24605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: No bootstrap required, opened a new log
I20260927 11:08:27.333289 24605 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } }
I20260927 11:08:27.333416 24605 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:27.333438 24605 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cba763d804494a926882ce62290ee2, State: Initialized, Role: FOLLOWER
I20260927 11:08:27.333580 24605 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } }
I20260927 11:08:27.333886 24605 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:27.333840 24610 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } } }
I20260927 11:08:27.333945 24610 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 11:08:27.335376 24621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:08:27.335430 24621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:08:27.337505 24537 master_runner.cc:428] Detected that this master f1cba763d804494a926882ce62290ee2 is joining an existing cluster
I20260927 11:08:27.337572 24537 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.95.252:33083
I20260927 11:08:27.345487 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.338368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.17.95.254:34341)
0927 11:08:27.338827 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.338829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.338836 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:27.338858 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.338859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.338868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.338924 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.339327 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.340062 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341281 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.341584 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341636 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.342939 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.342940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.342940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.342946 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.344437 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.344756 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.344756 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.344757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.344806 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.344997 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.344998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.344998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.345117 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.345184 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.345331 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.345398 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260927 11:08:27.345532 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.338349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:57672)
0927 11:08:27.338494 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.338507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.338562 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.338929 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.338930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.339174 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.339247 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.339248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.340140 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.340141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.341173 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.341175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.341696 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.342158 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.342542 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.342546 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.342839 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.344515 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.344516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.344517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.344692 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.344869 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.344870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.344870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.344923 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.345176 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.345176 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.345226 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.345308 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.345401 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":83,"thread_start_us":71,"threads_started":1}
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507306, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.254@KRBTEST.COM for oryx/127.17.95.252@KRBTEST.COM
I20260927 11:08:27.366061 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.348518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:56366)
0927 11:08:27.348752 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.348762 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.348813 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.349027 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.349028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.349185 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.349228 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.349230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.350122 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.350123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.350740 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.350742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.351148 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.351149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.351212 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.351580 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.351584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.352682 (+ 1098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.365040 (+ 12358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.365041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.365041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.365199 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.365416 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.365417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.365417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.365476 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.365738 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.365739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.365778 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.365842 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.365934 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260927 11:08:27.366598 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.348543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.17.95.252:33083)
0927 11:08:27.348708 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.348710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.348798 (+ 88us) server_negotiation.cc:407] Connection header received
0927 11:08:27.348917 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.348919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.348930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.349304 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.349339 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.350045 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.350828 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.351042 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.351146 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.352780 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.352781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.352782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.352787 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.364961 (+ 12174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.365275 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.365276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.365276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.365335 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.365559 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.365560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.365560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.365668 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.366275 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.366312 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.366462 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260927 11:08:27.367237 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.338986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.17.95.253:36361)
0927 11:08:27.339024 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.339026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.339032 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:08:27.339051 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.339051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.339060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.339129 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.339488 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.340169 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341265 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.341543 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341580 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.364113 (+ 22533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.364115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.364115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.364124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.365962 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.366374 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.366375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.366376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.366433 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.366651 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.366652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.366653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.366787 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.366827 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.367023 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.367127 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:27.367375 23884 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.95.252:33083: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f1cba763d804494a926882ce62290ee2"
member_type: NON_VOTER
last_known_addr {
host: "127.17.95.252"
port: 33083
}
attrs {
promote: true
}
}
dest_uuid: "65f7cffdc5d24037a5c02798d6935c27"
cas_config_opid_index: 5
I20260927 11:08:27.367616 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.338640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37666)
0927 11:08:27.338759 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.338768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.338812 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.339177 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.339178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.339359 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.339411 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.339412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.340268 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.340269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.341173 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.341175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.341650 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.341718 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.363522 (+ 21804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.363526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.364020 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.366038 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.366038 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.366039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.366276 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.366513 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.366514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.366514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.366579 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.366871 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.366872 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.366913 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.367441 (+ 528us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.367549 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":34,"thread_start_us":87,"threads_started":1}
I20260927 11:08:27.367605 23884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } }
I20260927 11:08:27.367926 24444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f1cba763d804494a926882ce62290ee2 (127.17.95.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.368345 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.338557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.17.95.252:33083)
0927 11:08:27.338590 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.338591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.338792 (+ 201us) server_negotiation.cc:407] Connection header received
0927 11:08:27.338882 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.338883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.338890 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.338931 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.339380 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.340058 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341215 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.341478 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341516 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.363929 (+ 22413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.363930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.363930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.363936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.366581 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.366919 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.366919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.366919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.366961 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.367172 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.367173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.367173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.367288 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.367326 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.368173 (+ 847us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.368261 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:08:27.368578 24579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:27.368990 24444 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.95.252:33083
I20260927 11:08:27.369077 24444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.369141 24444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:27.369551 24579 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 FOLLOWER]: Refusing update from remote peer 65f7cffdc5d24037a5c02798d6935c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 11:08:27.369828 24443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:27.370571 24537 master.cc:537] Master@127.17.95.252:33083 shutting down...
I20260927 11:08:27.371212 24579 raft_consensus.cc:858] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } }
W20260927 11:08:27.371309 24579 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } }
I20260927 11:08:27.371598 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.338625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:56358)
0927 11:08:27.338740 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.338750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.338813 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.338972 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.338972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.339235 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.339301 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.339303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.340141 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.340141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.341129 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.341131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.341602 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.341603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.341655 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.342109 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.342113 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.363838 (+ 21725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.366663 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.366664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.366664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.366849 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.367033 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.367033 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.367033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.367111 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.368018 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.368019 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.368074 (+ 55us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.371096 (+ 3022us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.371221 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260927 11:08:27.373601 24579 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d8d0490bd71446f48b3fee40f9f70383 (127.17.95.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.374742 24634 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.374879 24634 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 11:08:27.374851 24610 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f7cffdc5d24037a5c02798d6935c27. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.375404 24610 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 11:08:27.374974 24634 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } }
I20260927 11:08:27.375483 24634 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 11:08:27.375627 24631 mvcc.cc:205] Tried to move back new op lower bound from 7333917927033085952 to 7333917926921207808. Current Snapshot: MvccSnapshot[applied={T|T < 7333917926931824640}]
I20260927 11:08:27.376246 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.368127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:37678)
0927 11:08:27.370156 (+ 2029us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.370167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.370202 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.370409 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.370410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.370411 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:27.370444 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.370446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.372926 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.372927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.374138 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.374141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.376011 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.376012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.376080 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.376099 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.376148 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.376183 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2013}
I20260927 11:08:27.376648 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.368204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.17.95.253:36361)
0927 11:08:27.370266 (+ 2062us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.370268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.370274 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:08:27.370292 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.370293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.370313 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.370365 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:27.371907 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.372658 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.374235 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.375817 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.375887 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.375978 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.376549 (+ 571us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.376585 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2044}
I20260927 11:08:27.376693 24443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.376785 24443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:27.377032 24493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 LEARNER]: Refusing update from remote peer 65f7cffdc5d24037a5c02798d6935c27: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 11:08:27.377220 24444 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260927 11:08:27.377548 23871 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 -> Peer f1cba763d804494a926882ce62290ee2 (127.17.95.252:33083): Couldn't send request to peer f1cba763d804494a926882ce62290ee2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 11:08:27.377842 24537 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260927 11:08:27.377931 24537 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260927 11:08:27.377961 24537 tablet_replica.cc:330] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: stopping tablet replica
I20260927 11:08:27.378067 24493 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f1cba763d804494a926882ce62290ee2 (127.17.95.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.379235 24639 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f7cffdc5d24037a5c02798d6935c27. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.379240 24522 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: NON_VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: true } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.379336 24639 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: LEARNER
I20260927 11:08:27.379343 24522 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: LEARNER
I20260927 11:08:27.380115 24444 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: attempting to promote NON_VOTER d8d0490bd71446f48b3fee40f9f70383 to VOTER
I20260927 11:08:27.380294 24444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } }
I20260927 11:08:27.381141 24493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 LEARNER]: Refusing update from remote peer 65f7cffdc5d24037a5c02798d6935c27: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 11:08:27.381165 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.381078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:56380)
0927 11:08:27.381112 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.381136 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:33083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260927 11:08:27.381213 23871 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 -> Peer f1cba763d804494a926882ce62290ee2 (127.17.95.252:33083): Couldn't send request to peer f1cba763d804494a926882ce62290ee2. Status: Network error: Client connection negotiation failed: client connection to 127.17.95.252:33083: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 11:08:27.381315 24445 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 11:08:27.382004 24444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d8d0490bd71446f48b3fee40f9f70383 (127.17.95.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.382154 24493 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d8d0490bd71446f48b3fee40f9f70383 (127.17.95.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.382879 24444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.382899 24522 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.382942 24444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:27.382959 24522 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:08:27.382988 24444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.383038 24444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:27.384409 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.384279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:56388)
0927 11:08:27.384357 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.384391 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:33083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 11:08:27.394357 24537 master.cc:552] Master@127.17.95.252:33083 shutdown complete.
I20260927 11:08:27.394431 24537 master_runner.cc:305] Clearing existing system tablet
I20260927 11:08:27.394556 24537 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.394610 24537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.395315 24537 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:27.395812 24641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.395969 24537 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.396030 24537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.396060 24537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
uuid: "f1cba763d804494a926882ce62290ee2"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "22f932898810f737cd42103e28712713"
server_key_iv: "6357caba5e594399d2e7e52071391150"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.396581 24537 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:27.397728 24537 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 11:08:27.397799 24537 log.cc:1196] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 11:08:27.398028 24537 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Deleting consensus metadata
I20260927 11:08:27.398151 24537 master_runner.cc:315] Copying system tablet from 127.17.95.254:34341
I20260927 11:08:27.398763 24537 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: tablet copy: Beginning tablet copy session from remote peer at address 127.17.95.254:34341
I20260927 11:08:27.404716 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.399066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.17.95.254:34341)
0927 11:08:27.399132 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.399134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.399198 (+ 64us) server_negotiation.cc:407] Connection header received
0927 11:08:27.399256 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.399256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.399265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.399319 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.399566 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.400304 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.400950 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.401243 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.401304 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.402210 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.402211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.402211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.402218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.403665 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.403988 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.403989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.403989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.404043 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.404227 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.404228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.404229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.404332 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.404400 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.404541 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.404636 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:27.404729 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.399027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:57688)
0927 11:08:27.399136 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.399145 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.399202 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.399321 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.399322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.399459 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.399507 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.399508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.400377 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.400378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.400875 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.400876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.401315 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.401316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.401359 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.401821 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.401826 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.402128 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.403761 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.403762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.403762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.403923 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.404109 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.404110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.404110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.404164 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.404392 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.404393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.404435 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.404514 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.404630 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":77,"threads_started":1}
I20260927 11:08:27.404906 23904 tablet_copy_service.cc:140] P 65f7cffdc5d24037a5c02798d6935c27: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f1cba763d804494a926882ce62290ee2 ({username='oryx', principal='oryx/127.17.95.252@KRBTEST.COM'} at 127.0.0.1:57688)
I20260927 11:08:27.404946 23904 tablet_copy_service.cc:161] P 65f7cffdc5d24037a5c02798d6935c27: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f1cba763d804494a926882ce62290ee2 at {username='oryx', principal='oryx/127.17.95.252@KRBTEST.COM'} at 127.0.0.1:57688: session id = f1cba763d804494a926882ce62290ee2-00000000000000000000000000000000
I20260927 11:08:27.405308 23904 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 11:08:27.405692 24537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 11:08:27.406507 24537 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: tablet copy: Starting download of 0 data blocks...
I20260927 11:08:27.406633 24537 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: tablet copy: Starting download of 1 WAL segments...
I20260927 11:08:27.407358 24537 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 11:08:27.408042 23904 tablet_copy_service.cc:342] P 65f7cffdc5d24037a5c02798d6935c27: Request end of tablet copy session f1cba763d804494a926882ce62290ee2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.95.252@KRBTEST.COM'} at 127.0.0.1:57688
I20260927 11:08:27.408090 23904 tablet_copy_service.cc:433] P 65f7cffdc5d24037a5c02798d6935c27: ending tablet copy session f1cba763d804494a926882ce62290ee2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f1cba763d804494a926882ce62290ee2
I20260927 11:08:27.410403 24537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.411556 24651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.411718 24649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.411584 24648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.411789 24537 server_base.cc:1034] running on GCE node
I20260927 11:08:27.411921 24537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.412063 24537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.413194 24537 hybrid_clock.cc:648] HybridClock initialized: now 1790507307413190 us; error 26 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.414381 24537 init.cc:377] Logged in from keytab as oryx/127.17.95.252@KRBTEST.COM (short username oryx)
I20260927 11:08:27.414628 24537 webserver.cc:460] Webserver started at http://127.17.95.252:42823/ using document root <none> and password file <none>
I20260927 11:08:27.414738 24537 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.414783 24537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.415411 24537 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:27.415930 24658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.416082 24537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.416139 24537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
uuid: "f1cba763d804494a926882ce62290ee2"
format_stamp: "Formatted at 2026-09-27 11:08:27 on dist-test-slave-6j0v"
server_key: "22f932898810f737cd42103e28712713"
server_key_iv: "6357caba5e594399d2e7e52071391150"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.416193 24537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.448302 24537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.449970 24537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.452931 24537 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.252:33083
I20260927 11:08:27.452955 24710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.252:33083 every 8 connection(s)
I20260927 11:08:27.453234 24537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
I20260927 11:08:27.454128 24711 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:08:27.454639 24711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Bootstrap starting.
I20260927 11:08:27.457113 24711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:08:27.457330 24711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: Bootstrap complete.
I20260927 11:08:27.457538 24711 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } }
I20260927 11:08:27.457639 24711 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f1cba763d804494a926882ce62290ee2, State: Initialized, Role: LEARNER
I20260927 11:08:27.457680 24711 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } }
I20260927 11:08:27.457849 24711 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:27.457854 24714 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.457995 24714 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: LEARNER
I20260927 11:08:27.459380 24725 catalog_manager.cc:1259] Loaded cluster ID: a00062d2db18484596fba69a1bddb89b
I20260927 11:08:27.459429 24725 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: loading cluster ID for follower catalog manager: success
I20260927 11:08:27.460561 24725 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: acquiring CA information for follower catalog manager: success
I20260927 11:08:27.460816 24725 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:08:27.488601 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.482534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56408 (local address 127.17.95.252:33083)
0927 11:08:27.482667 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.482669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.482677 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:27.482699 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.482700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.482709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.482763 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.483379 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.484025 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.484712 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.484898 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.484951 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.486128 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.486129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.486129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.486143 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.487635 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.487962 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.487962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.487963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.488016 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.488205 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.488205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.488206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.488298 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.488383 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.488479 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.488535 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260927 11:08:27.489127 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23946
I20260927 11:08:27.494226 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:45643
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:34341
--webserver_port=46005
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:27.579758 24727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:27.580046 24727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:27.580142 24727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:27.581351 24727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:27.581421 24727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:27.581466 24727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:27.582971 24727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:45643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=46005
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:27.583251 24727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:27.583482 24727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.588712 24735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.588754 24733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.588831 24727 server_base.cc:1034] running on GCE node
W20260927 11:08:27.588712 24732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.589143 24727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.589377 24727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.590543 24727 hybrid_clock.cc:648] HybridClock initialized: now 1790507307590538 us; error 37 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.592963 24727 init.cc:377] Logged in from keytab as oryx/127.17.95.193@KRBTEST.COM (short username oryx)
I20260927 11:08:27.593358 24727 webserver.cc:460] Webserver started at http://127.17.95.193:46005/ using document root <none> and password file <none>
I20260927 11:08:27.593592 24727 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.593669 24727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.594980 24727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:27.595589 24742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.595786 24727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:08:27.595885 24727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "3ad61acf3e76459d93d1f114fdd47ab9"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "3314e727c75c51446ec123f9025dc6e7"
server_key_iv: "b9d32ada144a78a6df1903457ca5fea8"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.596290 24727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.608476 24727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.610468 24727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:27.610630 24727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.610857 24727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:27.611222 24727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:27.611276 24727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.611320 24727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:27.611352 24727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.618206 24727 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:45643
I20260927 11:08:27.618263 24855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:45643 every 8 connection(s)
I20260927 11:08:27.618615 24727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:27.618820 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24727
I20260927 11:08:27.618983 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24080
I20260927 11:08:27.625964 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:34313
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:34341
--webserver_port=38777
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.252@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
I20260927 11:08:27.631434 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.620069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:52381 (local address 127.17.95.252:33083)
0927 11:08:27.620117 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.620120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.622096 (+ 1976us) server_negotiation.cc:407] Connection header received
0927 11:08:27.622130 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.622131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.622145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.622220 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.622590 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.623398 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.624414 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.624751 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.624819 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.627631 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.627632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.627633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.627640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.629453 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.630353 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.630354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.630354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.630402 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.630686 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.630686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.630687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.630844 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.630887 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.631245 (+ 358us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.631347 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 11:08:27.631903 24860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.619584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.17.95.193:52381)
0927 11:08:27.622027 (+ 2443us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.622036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.622080 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.622238 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.622239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.622429 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.622496 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.622498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.623489 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.623490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.624340 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.624344 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.624824 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.624825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.624874 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.625522 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.625533 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.627518 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.629876 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.629877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.629877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.630263 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.630474 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.630475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.630475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.630607 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.630930 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.630933 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.631103 (+ 170us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.631549 (+ 446us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.631642 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2425,"thread_start_us":55,"threads_started":1}
I20260927 11:08:27.631906 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.619325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:51353)
0927 11:08:27.619763 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.619806 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.620531 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.620759 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.620764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.621619 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.621883 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.621889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.622916 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.622917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.624279 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.624282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.624701 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.624702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.624842 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.625564 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.625569 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.627788 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.629859 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.629866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.629868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.630263 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.630483 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.630484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.630484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.630607 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.630928 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.630933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.631079 (+ 146us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.631199 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.631365 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260927 11:08:27.632359 24856 heartbeater.cc:347] Connected to a master server at 127.17.95.252:33083
I20260927 11:08:27.632436 24856 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.632530 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.620099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:51353 (local address 127.17.95.254:34341)
0927 11:08:27.620152 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.620154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.620162 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:27.620630 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.620631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.620640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.620682 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.621990 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.622797 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.624365 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.624607 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.624691 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.627870 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.627871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.627871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.627878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.629763 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.630346 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.630347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.630348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.630403 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.630680 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.630681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.630681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.630845 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.630902 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.631971 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.632049 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 11:08:27.632612 24856 heartbeater.cc:511] Master 127.17.95.252:33083 requested a full tablet report, sending...
I20260927 11:08:27.632879 24857 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34341
I20260927 11:08:27.632931 24857 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.633001 24857 heartbeater.cc:511] Master 127.17.95.254:34341 requested a full tablet report, sending...
I20260927 11:08:27.633148 24675 ts_manager.cc:194] Registered new tserver with Master: 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643)
I20260927 11:08:27.633236 23884 ts_manager.cc:194] Re-registered known tserver with Master: 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643)
I20260927 11:08:27.633689 23884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:51353
I20260927 11:08:27.634362 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.623817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:58543 (local address 127.17.95.253:36361)
0927 11:08:27.623864 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.623865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.625827 (+ 1962us) server_negotiation.cc:407] Connection header received
0927 11:08:27.625924 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.625925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.625936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.625977 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.626403 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.627150 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.628184 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.628495 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.628567 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.631281 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.631283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.631283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.631290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.633021 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.633425 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.633426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.633426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.633486 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.633730 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.633732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.633732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.633884 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.633945 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.634188 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.634263 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:08:27.634423 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.623657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.17.95.193:58543)
0927 11:08:27.625757 (+ 2100us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.625770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.625854 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.626045 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.626046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.626239 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.626310 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.626312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.627269 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.627270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.628078 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.628081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.628820 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.628821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.628898 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.629597 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.629603 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.631160 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.633112 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.633113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.633113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.633334 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.633565 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.633566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.633567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.633654 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.634009 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.634010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.634056 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.634144 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.634255 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2079,"thread_start_us":64,"threads_started":1}
I20260927 11:08:27.635259 24858 heartbeater.cc:347] Connected to a master server at 127.17.95.253:36361
I20260927 11:08:27.635316 24858 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.635386 24858 heartbeater.cc:511] Master 127.17.95.253:36361 requested a full tablet report, sending...
I20260927 11:08:27.635800 24483 ts_manager.cc:194] Registered new tserver with Master: 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643)
W20260927 11:08:27.710026 24863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:27.710188 24863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:27.710208 24863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:27.711239 24863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:27.711272 24863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:27.711297 24863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:27.712625 24863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:34313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=38777
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:27.712832 24863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:27.713083 24863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.717836 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.717911 24869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.717868 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.718019 24863 server_base.cc:1034] running on GCE node
I20260927 11:08:27.718184 24863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.718364 24863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.719533 24863 hybrid_clock.cc:648] HybridClock initialized: now 1790507307719515 us; error 34 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.721804 24863 init.cc:377] Logged in from keytab as oryx/127.17.95.194@KRBTEST.COM (short username oryx)
I20260927 11:08:27.722241 24863 webserver.cc:460] Webserver started at http://127.17.95.194:38777/ using document root <none> and password file <none>
I20260927 11:08:27.722440 24863 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.722496 24863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.723847 24863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:08:27.724460 24878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.724653 24863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:27.724749 24863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "44404d6e189f4c81a451dae98db9dac7"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "4674ed1c450fb01011983d9016ae65d7"
server_key_iv: "c218933e07f681a84b7e7e57fe3c3abd"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.725121 24863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.739025 24863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.740772 24863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:27.740906 24863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.741124 24863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:27.741430 24863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:27.741479 24863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.741500 24863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:27.741513 24863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.747386 24863 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:34313
I20260927 11:08:27.747438 24991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:34313 every 8 connection(s)
I20260927 11:08:27.747792 24863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:27.752456 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24863
I20260927 11:08:27.752616 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24214
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.252@KRBTEST.COM
I20260927 11:08:27.759397 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:45481
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:34341
--webserver_port=44547
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:45031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:08:27.760128 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.748832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:40565 (local address 127.17.95.253:36361)
0927 11:08:27.748959 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.748961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.749208 (+ 247us) server_negotiation.cc:407] Connection header received
0927 11:08:27.749676 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.749677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.749689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.749720 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.750524 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.751054 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752246 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.752538 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752578 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.756126 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.756127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.756127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.756133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.757917 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759091 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.759092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.759092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.759145 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759392 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.759393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.759393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.759511 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.759555 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.759959 (+ 404us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.760043 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 11:08:27.760128 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.748475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:50505 (local address 127.17.95.252:33083)
0927 11:08:27.748804 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.748807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.748989 (+ 182us) server_negotiation.cc:407] Connection header received
0927 11:08:27.749677 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.749678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.749691 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.749752 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.751222 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.751910 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752940 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.753265 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.753316 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.756864 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.756866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.756866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.756875 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.758572 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759040 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.759041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.759041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.759115 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759393 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.759393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.759394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.759543 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.759582 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.759959 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.760038 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260927 11:08:27.760195 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.748898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:39369 (local address 127.17.95.254:34341)
0927 11:08:27.748956 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.748958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.749341 (+ 383us) server_negotiation.cc:407] Connection header received
0927 11:08:27.749676 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.749678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.749689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.749720 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.750564 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.751107 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752242 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.752527 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752585 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.755702 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.755704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.755704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.755711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.758332 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.758985 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.758986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.758986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.759051 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759399 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.759401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.759401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.759580 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.759631 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.759966 (+ 335us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.760054 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 11:08:27.760836 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.748445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.17.95.194:50505)
0927 11:08:27.748818 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.748877 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.749562 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.749841 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.749842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.750220 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.750418 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.750422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.752002 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.752815 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.752818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.753397 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.753398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.753478 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.754498 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.754502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.756767 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.758645 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.758646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.758646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.758949 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.759191 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.759192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.759302 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.759655 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.759661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.759810 (+ 149us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.760311 (+ 501us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.760607 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":156,"thread_start_us":78,"threads_started":1}
I20260927 11:08:27.760845 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.748726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:39369)
0927 11:08:27.749268 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.749277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.749586 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.749836 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.749840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.750308 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.750419 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.750425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.751222 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.751223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.752137 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.752139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.752642 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.752777 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.753354 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.753364 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.755612 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.758418 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.758425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.758427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.758883 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.759155 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.759157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.759301 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.759701 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.759702 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.759813 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.760303 (+ 490us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.760540 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":37,"thread_start_us":209,"threads_started":1}
I20260927 11:08:27.761374 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.748720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.17.95.194:40565)
0927 11:08:27.749149 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.749159 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.749581 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.749836 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.749840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.750224 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.750419 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.750425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.751209 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.751210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.752152 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.752154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.752719 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.752720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.752777 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.753450 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.753455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.756020 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.758692 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.758693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.758693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.758991 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.759217 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.759218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.759218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.759302 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.759788 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.759789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.759818 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.760702 (+ 884us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.760789 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":58,"thread_start_us":293,"threads_started":1}
I20260927 11:08:27.761380 24993 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34341
I20260927 11:08:27.761500 24993 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.761590 24992 heartbeater.cc:347] Connected to a master server at 127.17.95.252:33083
I20260927 11:08:27.761637 24992 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.761682 24993 heartbeater.cc:511] Master 127.17.95.254:34341 requested a full tablet report, sending...
I20260927 11:08:27.761689 24994 heartbeater.cc:347] Connected to a master server at 127.17.95.253:36361
I20260927 11:08:27.761718 24992 heartbeater.cc:511] Master 127.17.95.252:33083 requested a full tablet report, sending...
I20260927 11:08:27.761762 24994 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.761817 24994 heartbeater.cc:511] Master 127.17.95.253:36361 requested a full tablet report, sending...
I20260927 11:08:27.762138 24483 ts_manager.cc:194] Registered new tserver with Master: 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313)
I20260927 11:08:27.762161 24675 ts_manager.cc:194] Registered new tserver with Master: 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313)
I20260927 11:08:27.762290 23884 ts_manager.cc:194] Re-registered known tserver with Master: 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313)
I20260927 11:08:27.762748 23884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:39369
W20260927 11:08:27.841594 24999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:27.841809 24999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:27.841832 24999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:27.843047 24999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:27.843142 24999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:27.843194 24999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:27.844696 24999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:45031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:45481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=44547
--tserver_master_addrs=127.17.95.254:34341,127.17.95.253:36361,127.17.95.252:33083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:27.844937 24999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:27.845211 24999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:27.850097 25007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:27.850106 25004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.850133 24999 server_base.cc:1034] running on GCE node
W20260927 11:08:27.850106 25005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:27.850559 24999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:27.850826 24999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:27.851984 24999 hybrid_clock.cc:648] HybridClock initialized: now 1790507307851968 us; error 26 us; skew 500 ppm
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:27.854360 24999 init.cc:377] Logged in from keytab as oryx/127.17.95.195@KRBTEST.COM (short username oryx)
I20260927 11:08:27.854839 24999 webserver.cc:460] Webserver started at http://127.17.95.195:44547/ using document root <none> and password file <none>
I20260927 11:08:27.855057 24999 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:27.855170 24999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:27.856521 24999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:27.857120 25014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.857313 24999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:27.857407 24999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "facf2d136f2549d1872c3ec4d5705f53"
format_stamp: "Formatted at 2026-09-27 11:08:26 on dist-test-slave-6j0v"
server_key: "56c92054dd75d07330ab6d6adeb967d2"
server_key_iv: "4f354da7eda3706693e81efbec0b8a38"
server_key_version: "encryptionkey@0"
I20260927 11:08:27.857757 24999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:27.877310 24999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:27.879562 24999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:27.879714 24999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:27.879942 24999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:27.880249 24999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:27.880301 24999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.880362 24999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:27.880401 24999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:27.886266 24999 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:45481
I20260927 11:08:27.886307 25127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:45481 every 8 connection(s)
I20260927 11:08:27.886662 24999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.253@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
Sep 27 11:08:27 dist-test-slave-6j0v krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507307, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.252@KRBTEST.COM
I20260927 11:08:27.895339 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 24999
I20260927 11:08:27.897629 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.887753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:50291 (local address 127.17.95.254:34341)
0927 11:08:27.887802 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.887804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.887900 (+ 96us) server_negotiation.cc:407] Connection header received
0927 11:08:27.888402 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.888403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.888415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.888446 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.889102 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.889841 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.890907 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.891152 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.891211 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.894124 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.894125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.894126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.894134 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.895592 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896510 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.896511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.896511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.896586 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896894 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.896895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.896895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.897025 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.897069 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.897461 (+ 392us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.897540 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:27.897822 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.887861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.17.95.195:49723)
0927 11:08:27.888038 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.888047 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.888291 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.888499 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.888503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.888807 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.888993 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.888998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.889661 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.889661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.890432 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.890433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.890830 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.890831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.890979 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.891550 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.891559 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.893018 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.895701 (+ 2683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.895702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.895703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.896416 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.896654 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.896655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.896804 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.897105 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.897112 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.897293 (+ 181us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.897416 (+ 123us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.897585 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260927 11:08:27.898000 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.887698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.195:50291)
0927 11:08:27.887800 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.887810 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.888281 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.888515 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.888516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.888935 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.888991 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.888998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.889918 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.889918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.890824 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.890825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.891284 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.891285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.891349 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.891680 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.891684 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.894034 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.895680 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.895686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.895688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.896416 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.896652 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.896653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.896804 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.897117 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.897117 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.897293 (+ 176us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.897774 (+ 481us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.897866 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":72,"thread_start_us":58,"threads_started":1}
I20260927 11:08:27.898154 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.887373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:40729 (local address 127.17.95.252:33083)
0927 11:08:27.887417 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.887419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.887717 (+ 298us) server_negotiation.cc:407] Connection header received
0927 11:08:27.888401 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.888403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.888415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.888446 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.889101 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.889818 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.890922 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.891223 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.891316 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.895157 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.895158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.895158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.895164 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.896897 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.897284 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.897285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.897286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.897346 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.897562 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.897563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.897564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.897701 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.897748 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.897962 (+ 214us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.898067 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:27.898489 25129 heartbeater.cc:347] Connected to a master server at 127.17.95.254:34341
I20260927 11:08:27.898607 25129 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.898641 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.887920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:49723 (local address 127.17.95.253:36361)
0927 11:08:27.887968 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.887970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.888087 (+ 117us) server_negotiation.cc:407] Connection header received
0927 11:08:27.888375 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.888376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.888388 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.888431 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.889082 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.889563 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.890536 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.890743 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.890824 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.893126 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.893128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.893128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.893135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.894862 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896509 (+ 1647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.896510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.896511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.896586 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896884 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.896884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.896885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.897016 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.897079 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.898382 (+ 1303us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.898471 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:27.898794 25129 heartbeater.cc:511] Master 127.17.95.254:34341 requested a full tablet report, sending...
I20260927 11:08:27.898483 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.887405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.17.95.195:40729)
0927 11:08:27.887598 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.887632 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.888281 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.888515 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.888516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.888834 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:27.888992 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.888998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.889911 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.889911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.890800 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.890802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.891317 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.891317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.891383 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.891845 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:27.891850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:27.895024 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:27.896966 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.896967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.896968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.897194 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.897425 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:27.897426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:27.897426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:27.897484 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:27.897801 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:27.897802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:27.897848 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.898229 (+ 381us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.898330 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":26,"thread_start_us":114,"threads_started":1}
I20260927 11:08:27.899199 23884 ts_manager.cc:194] Re-registered known tserver with Master: facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481)
I20260927 11:08:27.899196 25130 heartbeater.cc:347] Connected to a master server at 127.17.95.253:36361
I20260927 11:08:27.899442 25130 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.899523 25130 heartbeater.cc:511] Master 127.17.95.253:36361 requested a full tablet report, sending...
I20260927 11:08:27.899737 23884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:50291
I20260927 11:08:27.899856 24483 ts_manager.cc:194] Registered new tserver with Master: facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481)
I20260927 11:08:27.900233 25128 heartbeater.cc:347] Connected to a master server at 127.17.95.252:33083
I20260927 11:08:27.900282 25128 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:27.900525 25128 heartbeater.cc:511] Master 127.17.95.252:33083 requested a full tablet report, sending...
I20260927 11:08:27.900851 24675 ts_manager.cc:194] Registered new tserver with Master: facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481)
I20260927 11:08:27.906342 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.897137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.17.95.254:34341)
0927 11:08:27.897701 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.897703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.898569 (+ 866us) server_negotiation.cc:407] Connection header received
0927 11:08:27.898657 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.898658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.898667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.898696 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.899564 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.900250 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.901413 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.901700 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.901771 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.903391 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.903392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.903393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.903400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.905070 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.905527 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.905528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.905529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.905582 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.905786 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.905787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.905787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.905901 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.906014 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.906133 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.906211 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":57,"threads_started":1}
I20260927 11:08:27.907387 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.896639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.17.95.253:36361)
0927 11:08:27.898322 (+ 1683us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.898324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.898332 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:27.898352 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.898353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.898365 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.898429 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.901057 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.901682 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.902615 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.902829 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.902925 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.904404 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.904405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.904405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.904413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.906084 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.906478 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.906479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.906479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.906544 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.906773 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.906774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.906775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.906910 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.907014 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.907168 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.907268 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1662,"thread_start_us":52,"threads_started":1}
I20260927 11:08:27.908898 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.897459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.17.95.252:33083)
0927 11:08:27.897695 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.897697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.900337 (+ 2640us) server_negotiation.cc:407] Connection header received
0927 11:08:27.900359 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.900360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.900370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.900434 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:27.901318 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.901878 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.903619 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.903935 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.903980 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.905414 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:27.905415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:27.905416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:27.905422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:27.907326 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.908171 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.908172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.908172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.908222 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:27.908425 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:27.908425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:27.908426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:27.908542 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:27.908635 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.908743 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.908813 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":63,"threads_started":1}
I20260927 11:08:27.908595 23884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:27.910430 23884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:27.917835 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.914175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:45481 (local address 127.0.0.1:47110)
0927 11:08:27.914520 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.914531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.914680 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.914863 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.914864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.914866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:27.914910 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.914911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.915794 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.915795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.916836 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.916838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.917598 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.917599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.917661 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.917666 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.917724 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.917763 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":11,"thread_start_us":73,"threads_started":1}
I20260927 11:08:27.917845 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.914288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.17.95.195:45481)
0927 11:08:27.914496 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.914501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.914649 (+ 148us) server_negotiation.cc:407] Connection header received
0927 11:08:27.914774 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.914777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.914793 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.914867 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:27.914963 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.915712 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.916915 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.917503 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.917596 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.917670 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.917737 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.917775 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260927 11:08:27.918494 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.914175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34313 (local address 127.0.0.1:52080)
0927 11:08:27.914324 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.914337 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.914392 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.914808 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.914809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.914812 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:27.914865 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.914867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.916211 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.916212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.917365 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.917367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.918268 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.918269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.918335 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.918339 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.918379 (+ 40us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.918404 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":86,"threads_started":1}
I20260927 11:08:27.918520 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.914476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52080 (local address 127.17.95.194:34313)
0927 11:08:27.914617 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.914621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.914632 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:27.914681 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.914683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.914702 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.914935 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:27.914994 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.916118 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.917485 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.918183 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.918252 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.918321 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.918402 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.918444 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260927 11:08:27.918915 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 6293f80cfa974be582e0f3317255edcf (DEFAULT_TABLE table=test-table [id=809c6b0577a8484b91c35a8fc96d55a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:27.919013 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.914356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45643 (local address 127.0.0.1:37612)
0927 11:08:27.914803 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.914813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.914854 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.915363 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.915364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.915366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:27.915402 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.915404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.916657 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.916658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.917705 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.917707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.918815 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.918816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.918874 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.918880 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.918909 (+ 29us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.918946 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":61,"threads_started":1}
I20260927 11:08:27.919272 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6293f80cfa974be582e0f3317255edcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:27.919176 24926 tablet_service.cc:1467] Processing CreateTablet for tablet 6293f80cfa974be582e0f3317255edcf (DEFAULT_TABLE table=test-table [id=809c6b0577a8484b91c35a8fc96d55a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:27.919397 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.914646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.17.95.193:45643)
0927 11:08:27.915195 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.915200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.915209 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:27.915247 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.915249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.915265 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.915463 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:27.915527 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.916567 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.917796 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.918720 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.919178 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.919258 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.919285 (+ 27us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.919325 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":68,"threads_started":1}
I20260927 11:08:27.919510 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6293f80cfa974be582e0f3317255edcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:27.919996 24790 tablet_service.cc:1467] Processing CreateTablet for tablet 6293f80cfa974be582e0f3317255edcf (DEFAULT_TABLE table=test-table [id=809c6b0577a8484b91c35a8fc96d55a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:27.920320 24790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6293f80cfa974be582e0f3317255edcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:27.921996 25157 tablet_bootstrap.cc:492] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Bootstrap starting.
I20260927 11:08:27.922789 25158 tablet_bootstrap.cc:492] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Bootstrap starting.
I20260927 11:08:27.923265 25157 tablet_bootstrap.cc:654] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:27.923465 25158 tablet_bootstrap.cc:654] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:27.923591 25157 log.cc:824] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:27.923754 25158 log.cc:824] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:27.924157 25159 tablet_bootstrap.cc:492] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Bootstrap starting.
I20260927 11:08:27.924384 25157 tablet_bootstrap.cc:492] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: No bootstrap required, opened a new log
I20260927 11:08:27.924464 25157 ts_tablet_manager.cc:1397] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 11:08:27.924575 25158 tablet_bootstrap.cc:492] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: No bootstrap required, opened a new log
I20260927 11:08:27.924646 25158 ts_tablet_manager.cc:1397] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:27.924726 25159 tablet_bootstrap.cc:654] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:27.925076 25159 log.cc:824] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:27.925798 25159 tablet_bootstrap.cc:492] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: No bootstrap required, opened a new log
I20260927 11:08:27.925868 25159 ts_tablet_manager.cc:1397] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:27.926786 25157 raft_consensus.cc:348] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:27.926934 25157 raft_consensus.cc:374] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:27.926976 25157 raft_consensus.cc:729] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44404d6e189f4c81a451dae98db9dac7, State: Initialized, Role: FOLLOWER
I20260927 11:08:27.927110 25157 consensus_queue.cc:260] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:27.927167 25158 raft_consensus.cc:348] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:27.927341 25158 raft_consensus.cc:374] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:27.927383 25158 raft_consensus.cc:729] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: facf2d136f2549d1872c3ec4d5705f53, State: Initialized, Role: FOLLOWER
I20260927 11:08:27.927398 25157 ts_tablet_manager.cc:1428] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:08:27.927536 24993 heartbeater.cc:503] Master 127.17.95.254:34341 was elected leader, sending a full tablet report...
I20260927 11:08:27.927493 25158 consensus_queue.cc:260] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:27.927799 25158 ts_tablet_manager.cc:1428] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:08:27.928051 25129 heartbeater.cc:503] Master 127.17.95.254:34341 was elected leader, sending a full tablet report...
I20260927 11:08:27.929355 25159 raft_consensus.cc:348] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:27.929466 25159 raft_consensus.cc:374] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:27.929484 25159 raft_consensus.cc:729] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad61acf3e76459d93d1f114fdd47ab9, State: Initialized, Role: FOLLOWER
I20260927 11:08:27.929615 25159 consensus_queue.cc:260] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:27.929865 25159 ts_tablet_manager.cc:1428] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260927 11:08:27.929870 24857 heartbeater.cc:503] Master 127.17.95.254:34341 was elected leader, sending a full tablet report...
I20260927 11:08:27.947744 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.945027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.17.95.252:33083)
0927 11:08:27.945091 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:27.945093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:27.945101 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:27.945125 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:27.945126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:27.945145 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:27.945214 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:27.945368 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.946056 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.946938 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.947442 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.947535 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.947595 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:27.947652 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 11:08:27.947686 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:27.947744 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:27.944933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:56434)
0927 11:08:27.944999 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:27.945011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:27.945060 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:27.945244 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:27.945246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:27.945248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:27.945294 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.945296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.946142 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.946143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.946856 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:27.946858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:27.947547 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:27.947548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:27.947598 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:27.947602 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:27.947646 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 11:08:27.947686 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 11:08:27.948062 24685 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260927 11:08:27.948491 24714 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f7cffdc5d24037a5c02798d6935c27. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:27.948989 24714 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: LEARNER
W20260927 11:08:27.999429 24996 tablet.cc:2369] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:28.002636 25164 raft_consensus.cc:482] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:28.002739 25164 raft_consensus.cc:504] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:28.003173 25164 leader_election.cc:288] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313), 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643)
I20260927 11:08:28.006126 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.003455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:41791 (local address 127.17.95.194:34313)
0927 11:08:28.003516 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:28.003518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:28.003555 (+ 37us) server_negotiation.cc:407] Connection header received
0927 11:08:28.003629 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:28.003630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:28.003639 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:28.003720 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:28.003876 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.004425 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.005362 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.005751 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.005833 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.005842 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:28.006014 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 11:08:28.006055 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:08:28.006135 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.003381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34313 (local address 127.17.95.195:41791)
0927 11:08:28.003492 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:28.003507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:28.003574 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:28.003727 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:28.003728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:28.003730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:28.003799 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:28.003800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.004516 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.004517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:28.005249 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:28.005251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.005840 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.005841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:28.005928 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.005930 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:28.006006 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 11:08:28.006046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 11:08:28.006127 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.003590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:59787 (local address 127.17.95.193:45643)
0927 11:08:28.003635 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:28.003637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:28.003645 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:28.003730 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:28.003731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:28.003739 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:28.003809 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:28.003961 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.004499 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.005371 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.005748 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.005836 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.005844 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:28.006013 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 11:08:28.006053 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:28.006155 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.003552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45643 (local address 127.17.95.195:59787)
0927 11:08:28.003605 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:28.003616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:28.003663 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:28.003813 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:28.003813 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:28.003815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:28.003889 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:28.003890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.004585 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.004586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:28.005274 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:28.005276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.005835 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.005836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:28.005923 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.005926 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:28.005992 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 11:08:28.006033 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 11:08:28.006517 24946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6293f80cfa974be582e0f3317255edcf" candidate_uuid: "facf2d136f2549d1872c3ec4d5705f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44404d6e189f4c81a451dae98db9dac7" is_pre_election: true
I20260927 11:08:28.006515 24810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6293f80cfa974be582e0f3317255edcf" candidate_uuid: "facf2d136f2549d1872c3ec4d5705f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" is_pre_election: true
I20260927 11:08:28.006677 24946 raft_consensus.cc:2454] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate facf2d136f2549d1872c3ec4d5705f53 in term 0.
I20260927 11:08:28.006680 24810 raft_consensus.cc:2454] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate facf2d136f2549d1872c3ec4d5705f53 in term 0.
I20260927 11:08:28.006898 25016 leader_election.cc:302] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ad61acf3e76459d93d1f114fdd47ab9, facf2d136f2549d1872c3ec4d5705f53; no voters:
I20260927 11:08:28.007059 25164 raft_consensus.cc:2781] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:28.007162 25164 raft_consensus.cc:482] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:28.007215 25164 raft_consensus.cc:3037] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:28.007931 25164 raft_consensus.cc:504] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:28.008066 25164 leader_election.cc:288] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [CANDIDATE]: Term 1 election: Requested vote from peers 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313), 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643)
I20260927 11:08:28.008242 24946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6293f80cfa974be582e0f3317255edcf" candidate_uuid: "facf2d136f2549d1872c3ec4d5705f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44404d6e189f4c81a451dae98db9dac7"
I20260927 11:08:28.008322 24946 raft_consensus.cc:3037] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:28.008309 24810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6293f80cfa974be582e0f3317255edcf" candidate_uuid: "facf2d136f2549d1872c3ec4d5705f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ad61acf3e76459d93d1f114fdd47ab9"
I20260927 11:08:28.008432 24810 raft_consensus.cc:3037] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:28.009048 24946 raft_consensus.cc:2454] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate facf2d136f2549d1872c3ec4d5705f53 in term 1.
I20260927 11:08:28.009189 24810 raft_consensus.cc:2454] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate facf2d136f2549d1872c3ec4d5705f53 in term 1.
I20260927 11:08:28.009199 25016 leader_election.cc:302] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44404d6e189f4c81a451dae98db9dac7, facf2d136f2549d1872c3ec4d5705f53; no voters:
I20260927 11:08:28.009338 25164 raft_consensus.cc:2781] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:28.009538 25164 raft_consensus.cc:686] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 1 LEADER]: Becoming Leader. State: Replica: facf2d136f2549d1872c3ec4d5705f53, State: Running, Role: LEADER
I20260927 11:08:28.009635 25164 consensus_queue.cc:237] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:28.010704 23884 catalog_manager.cc:5495] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 reported cstate change: term changed from 0 to 1, leader changed from <none> to facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "facf2d136f2549d1872c3ec4d5705f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:28.011300 25147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: NON_VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: true } } }
I20260927 11:08:28.011408 25147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:28.012176 25147 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: attempting to promote NON_VOTER f1cba763d804494a926882ce62290ee2 to VOTER
I20260927 11:08:28.012287 25147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } }
I20260927 11:08:28.012852 24493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Refusing update from remote peer 65f7cffdc5d24037a5c02798d6935c27: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 11:08:28.012881 24685 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 LEARNER]: Refusing update from remote peer 65f7cffdc5d24037a5c02798d6935c27: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 11:08:28.013079 25172 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260927 11:08:28.013195 25173 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260927 11:08:28.013732 25172 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f1cba763d804494a926882ce62290ee2 (127.17.95.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.013899 24493 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f1cba763d804494a926882ce62290ee2 (127.17.95.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.014050 24685 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f1cba763d804494a926882ce62290ee2 (127.17.95.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.014730 25172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.014827 25172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:28.014871 25147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.014937 25147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:28.014946 24714 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.015015 24714 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:08:28.014786 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.015053 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:28.015237 25175 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65f7cffdc5d24037a5c02798d6935c27" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:28.015345 25175 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:08:28.037940 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.035660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.17.95.195:45481)
0927 11:08:28.035746 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:28.035748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:28.035757 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:28.035790 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:28.035791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:28.035808 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:28.035877 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:28.035993 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.036485 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.037235 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.037423 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.037480 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.037636 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:28.037732 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:28.037792 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:28.037820 (+ 28us) server_negotiation.cc:299] Negotiation successful
0927 11:08:28.037837 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 11:08:28.039888 24946 raft_consensus.cc:1261] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Refusing update from remote peer facf2d136f2549d1872c3ec4d5705f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:28.039888 24810 raft_consensus.cc:1261] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 FOLLOWER]: Refusing update from remote peer facf2d136f2549d1872c3ec4d5705f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:28.040278 25164 consensus_queue.cc:1035] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:28.040414 25170 consensus_queue.cc:1035] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:28.043277 25179 mvcc.cc:205] Tried to move back new op lower bound from 7333917933728669696 to 7333917933608087552. Current Snapshot: MvccSnapshot[applied={T|T < 7333917933728669696}]
I20260927 11:08:28.043502 25178 mvcc.cc:205] Tried to move back new op lower bound from 7333917933728669696 to 7333917933608087552. Current Snapshot: MvccSnapshot[applied={T|T < 7333917933728669696}]
I20260927 11:08:28.045773 23883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:28.045878 23883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:28.047291 23883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65f7cffdc5d24037a5c02798d6935c27: Sending DeleteTablet for 3 replicas of tablet 6293f80cfa974be582e0f3317255edcf
I20260927 11:08:28.047626 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet 6293f80cfa974be582e0f3317255edcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:28 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:52080
I20260927 11:08:28.047611 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet 6293f80cfa974be582e0f3317255edcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:28 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:47110
I20260927 11:08:28.047833 25187 tablet_replica.cc:330] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: stopping tablet replica
I20260927 11:08:28.047930 25187 raft_consensus.cc:2229] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:28.048027 25187 raft_consensus.cc:2258] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:28.048329 24790 tablet_service.cc:1514] Processing DeleteTablet for tablet 6293f80cfa974be582e0f3317255edcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:28 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:37612
I20260927 11:08:28.048367 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 23852
I20260927 11:08:28.048475 25187 ts_tablet_manager.cc:1905] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:28.049410 25188 tablet_replica.cc:330] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: stopping tablet replica
I20260927 11:08:28.049561 25188 raft_consensus.cc:2229] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:28.049707 25188 raft_consensus.cc:2258] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:28.050086 25187 ts_tablet_manager.cc:1918] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:28.050161 25187 log.cc:1196] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/6293f80cfa974be582e0f3317255edcf
I20260927 11:08:28.050304 25188 ts_tablet_manager.cc:1905] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:28.050427 25187 ts_tablet_manager.cc:1939] T 6293f80cfa974be582e0f3317255edcf P 44404d6e189f4c81a451dae98db9dac7: Deleting consensus metadata
I20260927 11:08:28.050809 25186 tablet_replica.cc:330] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: stopping tablet replica
I20260927 11:08:28.050904 25186 raft_consensus.cc:2229] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:28.051019 25186 raft_consensus.cc:2258] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:28.052526 25188 ts_tablet_manager.cc:1918] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:28.052603 25188 log.cc:1196] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/6293f80cfa974be582e0f3317255edcf
I20260927 11:08:28.052840 25188 ts_tablet_manager.cc:1939] T 6293f80cfa974be582e0f3317255edcf P 3ad61acf3e76459d93d1f114fdd47ab9: Deleting consensus metadata
W20260927 11:08:28.055050 24662 connection.cc:535] server connection from 127.0.0.1:56434 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 11:08:28.055117 24746 connection.cc:535] server connection from 127.0.0.1:37612 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 11:08:28.055299 24882 connection.cc:535] server connection from 127.0.0.1:52080 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:08:28.055848 25186 ts_tablet_manager.cc:1905] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:28.057341 25186 ts_tablet_manager.cc:1918] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:28.057415 25186 log.cc:1196] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/6293f80cfa974be582e0f3317255edcf
I20260927 11:08:28.057703 25186 ts_tablet_manager.cc:1939] T 6293f80cfa974be582e0f3317255edcf P facf2d136f2549d1872c3ec4d5705f53: Deleting consensus metadata
W20260927 11:08:28.057996 25015 connection.cc:441] server connection from 127.0.0.1:47110 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47110 (request call id 1) could send its response
I20260927 11:08:28.064790 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.056773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.17.95.253:36361)
0927 11:08:28.056817 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:28.056819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:28.056985 (+ 166us) server_negotiation.cc:407] Connection header received
0927 11:08:28.057108 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:28.057109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:28.057136 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:28.057169 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:28.057945 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.058777 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.059906 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.060174 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.060235 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.061878 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:28.061879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:28.061879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:28.061886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:28.063524 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:28.063876 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:28.063877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:28.063878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:28.063931 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:28.064148 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:28.064149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:28.064150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:28.064287 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:28.064385 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:28.064585 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 11:08:28.064684 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:08:28.065135 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:28.056814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.17.95.252:33083)
0927 11:08:28.056854 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:28.056856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:28.057048 (+ 192us) server_negotiation.cc:407] Connection header received
0927 11:08:28.057155 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:28.057156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:28.057166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:28.057196 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:28.058543 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.059227 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.059947 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:28.060272 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:28.060335 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:28.062180 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:28.062182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:28.062182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:28.062189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:28.063903 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:28.064232 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:28.064234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:28.064234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:28.064284 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:28.064478 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:28.064479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:28.064480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:28.064598 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:28.064761 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:28.064948 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 11:08:28.065022 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:29.044605 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.044256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:47771)
0927 11:08:29.044427 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.044544 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260927 11:08:29.044618 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.044257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.195:52227)
0927 11:08:29.044483 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.044561 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
W20260927 11:08:29.045051 25129 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:34341 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
W20260927 11:08:29.045082 24993 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:34341 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.045465 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.045102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:38603)
0927 11:08:29.045351 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.045418 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":54,"threads_started":1}
W20260927 11:08:29.045679 24857 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:34341 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.548288 25199 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 65f7cffdc5d24037a5c02798d6935c27)
I20260927 11:08:29.548461 25199 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } }
I20260927 11:08:29.548717 25199 leader_election.cc:288] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341), f1cba763d804494a926882ce62290ee2 (127.17.95.252:33083)
I20260927 11:08:29.549077 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.548867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40266)
0927 11:08:29.548965 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.549018 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":60,"threads_started":1}
W20260927 11:08:29.549235 24470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.549476 25204 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 65f7cffdc5d24037a5c02798d6935c27)
I20260927 11:08:29.549628 25204 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } }
I20260927 11:08:29.549677 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.549584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40282)
0927 11:08:29.549623 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.549650 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 11:08:29.549752 24470 leader_election.cc:334] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341): Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.549901 25204 leader_election.cc:288] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341), d8d0490bd71446f48b3fee40f9f70383 (127.17.95.253:36361)
I20260927 11:08:29.550179 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.549950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40284)
0927 11:08:29.550106 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.550146 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":42,"threads_started":1}
W20260927 11:08:29.550370 24662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.550881 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.550783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40288)
0927 11:08:29.550833 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.550857 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 11:08:29.551025 24662 leader_election.cc:334] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341): Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.552557 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.549055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:33083 (local address 127.0.0.1:48758)
0927 11:08:29.549165 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.549192 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.549245 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.549485 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.549486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.549489 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.549553 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.549555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.550497 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.550497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.551654 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.551656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.552371 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.552372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.552420 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.552424 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.552479 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.552514 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":50,"threads_started":1}
I20260927 11:08:29.552608 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.549231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.17.95.252:33083)
0927 11:08:29.549335 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.549337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.549348 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:29.549375 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.549376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.549392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.549494 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.549644 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.550413 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.551726 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.552286 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.552361 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.552372 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.552495 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.552535 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":49,"threads_started":1}
I20260927 11:08:29.552837 24685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8d0490bd71446f48b3fee40f9f70383" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f1cba763d804494a926882ce62290ee2" is_pre_election: true
I20260927 11:08:29.552968 24685 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8d0490bd71446f48b3fee40f9f70383 in term 1.
I20260927 11:08:29.553113 24470 leader_election.cc:302] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8d0490bd71446f48b3fee40f9f70383, f1cba763d804494a926882ce62290ee2; no voters: 65f7cffdc5d24037a5c02798d6935c27
I20260927 11:08:29.553246 25199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 11:08:29.553306 25199 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 65f7cffdc5d24037a5c02798d6935c27)
I20260927 11:08:29.553334 25199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 1 FOLLOWER]: Advancing to term 2
I20260927 11:08:29.553566 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.550075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.17.95.253:36361)
0927 11:08:29.550301 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.550303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.550311 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:29.550332 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.550333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.550342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.550402 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.550545 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.551362 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.552745 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.553257 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.553333 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.553343 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.553471 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.553511 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260927 11:08:29.553565 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.550045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:36361 (local address 127.0.0.1:39274)
0927 11:08:29.550204 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.550214 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.550266 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.550402 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.550403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.550405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.550450 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.550452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.551452 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.551453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.552561 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.552563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.553334 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.553334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.553397 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.553400 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.553456 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.553499 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":46,"threads_started":1}
I20260927 11:08:29.554111 25199 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } }
I20260927 11:08:29.554251 25199 leader_election.cc:288] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [CANDIDATE]: Term 2 election: Requested vote from peers 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341), f1cba763d804494a926882ce62290ee2 (127.17.95.252:33083)
I20260927 11:08:29.554443 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.554360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40300)
0927 11:08:29.554396 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.554424 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260927 11:08:29.554512 24685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8d0490bd71446f48b3fee40f9f70383" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f1cba763d804494a926882ce62290ee2"
I20260927 11:08:29.554579 24685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 1 FOLLOWER]: Advancing to term 2
I20260927 11:08:29.554553 24493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1cba763d804494a926882ce62290ee2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8d0490bd71446f48b3fee40f9f70383" is_pre_election: true
I20260927 11:08:29.554662 24493 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1cba763d804494a926882ce62290ee2 in current term 2: Already voted for candidate d8d0490bd71446f48b3fee40f9f70383 in this term.
I20260927 11:08:29.554746 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.554649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40316)
0927 11:08:29.554699 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.554720 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 11:08:29.554798 24470 leader_election.cc:334] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341): Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
I20260927 11:08:29.554874 24661 leader_election.cc:302] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1cba763d804494a926882ce62290ee2; no voters: 65f7cffdc5d24037a5c02798d6935c27, d8d0490bd71446f48b3fee40f9f70383
I20260927 11:08:29.555370 24685 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8d0490bd71446f48b3fee40f9f70383 in term 2.
I20260927 11:08:29.555431 25204 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:08:29.555557 24470 leader_election.cc:302] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8d0490bd71446f48b3fee40f9f70383, f1cba763d804494a926882ce62290ee2; no voters: 65f7cffdc5d24037a5c02798d6935c27
I20260927 11:08:29.555660 25199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 11:08:29.555776 25199 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [term 2 LEADER]: Becoming Leader. State: Replica: d8d0490bd71446f48b3fee40f9f70383, State: Running, Role: LEADER
I20260927 11:08:29.555830 25199 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } }
I20260927 11:08:29.556118 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8d0490bd71446f48b3fee40f9f70383. Latest consensus state: current_term: 2 leader_uuid: "d8d0490bd71446f48b3fee40f9f70383" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:29.556190 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:29.556341 25211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:29.556823 25211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:29.556911 25211 catalog_manager.cc:1259] Loaded cluster ID: a00062d2db18484596fba69a1bddb89b
I20260927 11:08:29.556943 25211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:29.557118 25211 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:29.557215 25211 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Loaded TSK: 0
I20260927 11:08:29.557358 25211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:29.596280 24482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:29.597960 24482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:29.598978 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.598878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40332)
0927 11:08:29.598922 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.598948 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 11:08:29.599161 24685 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [term 2 FOLLOWER]: Refusing update from remote peer d8d0490bd71446f48b3fee40f9f70383: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260927 11:08:29.599315 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.599196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.0.0.1:40342)
0927 11:08:29.599270 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.599293 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260927 11:08:29.599450 24470 consensus_peers.cc:487] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 -> Peer 65f7cffdc5d24037a5c02798d6935c27 (127.17.95.254:34341): Couldn't send request to peer 65f7cffdc5d24037a5c02798d6935c27. Status: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 11:08:29.599476 25209 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260927 11:08:29.600183 25204 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8d0490bd71446f48b3fee40f9f70383. Latest consensus state: current_term: 2 leader_uuid: "d8d0490bd71446f48b3fee40f9f70383" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:29.600283 25204 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:08:29.600600 25212 mvcc.cc:205] Tried to move back new op lower bound from 7333917940115431424 to 7333917939940872192. Current Snapshot: MvccSnapshot[applied={T|T < 7333917940115431424}]
I20260927 11:08:29.600679 25199 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d8d0490bd71446f48b3fee40f9f70383" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:29.600776 25199 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:29.600847 25204 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d8d0490bd71446f48b3fee40f9f70383" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65f7cffdc5d24037a5c02798d6935c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34341 } } peers { permanent_uuid: "d8d0490bd71446f48b3fee40f9f70383" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 36361 } attrs { promote: false } } peers { permanent_uuid: "f1cba763d804494a926882ce62290ee2" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 33083 } attrs { promote: false } } }
I20260927 11:08:29.600925 25204 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cba763d804494a926882ce62290ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:08:29.605721 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.602638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34313 (local address 127.0.0.1:58068)
0927 11:08:29.602678 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.602690 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.602731 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.603166 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.603167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.603168 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.603197 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.603198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.604039 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.604040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.604714 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.604716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.605434 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.605499 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.605504 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.605563 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.605620 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260927 11:08:29.605882 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.602601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:45481 (local address 127.0.0.1:40590)
0927 11:08:29.602631 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.602642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.602691 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.602923 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.602924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.602926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.602991 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.602993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.603909 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.603910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.604899 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.604900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.605596 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.605692 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.605697 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.605752 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.605799 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":3}
I20260927 11:08:29.606003 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.602732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45643 (local address 127.0.0.1:56916)
0927 11:08:29.602840 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.602849 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.602888 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.603033 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.603034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.603036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.603080 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.603081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.603772 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.603773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.604891 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.604893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.605717 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.605778 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.605781 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.605832 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.605874 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":4,"thread_start_us":69,"threads_started":1}
I20260927 11:08:29.606001 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.602807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.17.95.193:45643)
0927 11:08:29.602912 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.602914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.602923 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:29.602949 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.602950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.602960 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.603019 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.603175 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.603693 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.604987 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.605565 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605696 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.605710 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.605867 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.605914 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
I20260927 11:08:29.606200 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.602893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.17.95.194:34313)
0927 11:08:29.603005 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.603007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.603015 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:29.603041 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.603042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.603053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.603273 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.603310 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.603948 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.604820 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.605352 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605430 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.605441 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.606078 (+ 637us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.606129 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":71,"threads_started":1}
I20260927 11:08:29.606279 24790 tablet_service.cc:1467] Processing CreateTablet for tablet 64509167a1754d6c99162899e5ee5505 (DEFAULT_TABLE table=test-table [id=94b741836a1b412792b6befcb2ca01ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:29.606387 24926 tablet_service.cc:1467] Processing CreateTablet for tablet 64509167a1754d6c99162899e5ee5505 (DEFAULT_TABLE table=test-table [id=94b741836a1b412792b6befcb2ca01ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:29.606413 24790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64509167a1754d6c99162899e5ee5505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:29.606415 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.602660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.17.95.195:45481)
0927 11:08:29.602773 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.602775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.602783 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:29.602810 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.602811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.602823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.603063 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.603129 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.603834 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605010 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.605508 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.605584 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.605593 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.606296 (+ 703us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.606348 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260927 11:08:29.606623 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64509167a1754d6c99162899e5ee5505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:29.607460 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 64509167a1754d6c99162899e5ee5505 (DEFAULT_TABLE table=test-table [id=94b741836a1b412792b6befcb2ca01ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:29.607584 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64509167a1754d6c99162899e5ee5505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:29.607893 25222 tablet_bootstrap.cc:492] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9: Bootstrap starting.
I20260927 11:08:29.608239 25222 tablet_bootstrap.cc:654] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:29.608881 25222 tablet_bootstrap.cc:492] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9: No bootstrap required, opened a new log
I20260927 11:08:29.608949 25222 ts_tablet_manager.cc:1397] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:29.609045 25224 tablet_bootstrap.cc:492] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Bootstrap starting.
I20260927 11:08:29.609097 25222 raft_consensus.cc:348] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.609164 25222 raft_consensus.cc:374] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:29.609189 25222 raft_consensus.cc:729] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad61acf3e76459d93d1f114fdd47ab9, State: Initialized, Role: FOLLOWER
I20260927 11:08:29.609254 25222 consensus_queue.cc:260] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.609404 25224 tablet_bootstrap.cc:654] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:29.609409 25222 ts_tablet_manager.cc:1428] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:29.609954 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.609736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:34351)
0927 11:08:29.609872 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.609913 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260927 11:08:29.610095 25228 tablet_bootstrap.cc:492] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Bootstrap starting.
I20260927 11:08:29.610445 25228 tablet_bootstrap.cc:654] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:29.610684 24858 heartbeater.cc:503] Master 127.17.95.253:36361 was elected leader, sending a full tablet report...
I20260927 11:08:29.610737 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.610642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:60135)
0927 11:08:29.610683 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.610715 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 11:08:29.610844 25224 tablet_bootstrap.cc:492] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: No bootstrap required, opened a new log
I20260927 11:08:29.610899 25224 ts_tablet_manager.cc:1397] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:29.611016 25224 raft_consensus.cc:348] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.611123 25224 raft_consensus.cc:374] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:29.611158 25224 raft_consensus.cc:729] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: facf2d136f2549d1872c3ec4d5705f53, State: Initialized, Role: FOLLOWER
I20260927 11:08:29.611214 25224 consensus_queue.cc:260] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.611359 25228 tablet_bootstrap.cc:492] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: No bootstrap required, opened a new log
I20260927 11:08:29.611362 25224 ts_tablet_manager.cc:1428] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:29.611415 25228 ts_tablet_manager.cc:1397] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:29.611531 25228 raft_consensus.cc:348] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.611606 25228 raft_consensus.cc:374] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:29.611636 25228 raft_consensus.cc:729] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44404d6e189f4c81a451dae98db9dac7, State: Initialized, Role: FOLLOWER
I20260927 11:08:29.611697 25228 consensus_queue.cc:260] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.611855 25228 ts_tablet_manager.cc:1428] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:29.611891 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.611679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.195:34175)
0927 11:08:29.611807 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.611858 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260927 11:08:29.612249 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.612078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:48453)
0927 11:08:29.612185 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.612218 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":69,"threads_started":1}
I20260927 11:08:29.612638 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.612552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:41545)
0927 11:08:29.612583 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.612613 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260927 11:08:29.612691 24994 heartbeater.cc:503] Master 127.17.95.253:36361 was elected leader, sending a full tablet report...
I20260927 11:08:29.618335 25225 raft_consensus.cc:482] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:29.618425 25225 raft_consensus.cc:504] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.618700 25225 leader_election.cc:288] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481), 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313)
I20260927 11:08:29.621290 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.618848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:38641 (local address 127.17.95.195:45481)
0927 11:08:29.618882 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.618883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.618891 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:29.618951 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.618952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.618960 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.619041 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.619204 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.619763 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.620623 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.620966 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.621049 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.621057 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.621208 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.621240 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:29.621290 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.618794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:45481 (local address 127.17.95.193:38641)
0927 11:08:29.618839 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.618848 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.618894 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.619044 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.619045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.619046 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.619127 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.619129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.619839 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.619840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.620539 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.620541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.621049 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.621050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.621120 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.621123 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.621195 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.621231 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":3}
I20260927 11:08:29.621374 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.619000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45873 (local address 127.17.95.194:34313)
0927 11:08:29.619034 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.619036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.619180 (+ 144us) server_negotiation.cc:407] Connection header received
0927 11:08:29.619224 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.619225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.619233 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.619293 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:29.619423 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.619867 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.620707 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.621038 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.621123 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.621132 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.621292 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.621322 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:08:29.621392 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.618834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34313 (local address 127.17.95.193:45873)
0927 11:08:29.619112 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.619121 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:29.619171 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:29.619305 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:29.619305 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:29.619307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:29.619353 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.619355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.619953 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.619954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.620607 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:29.620608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.621146 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.621147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:29.621211 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.621214 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:29.621284 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 11:08:29.621319 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":78,"threads_started":1}
I20260927 11:08:29.621560 25082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64509167a1754d6c99162899e5ee5505" candidate_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "facf2d136f2549d1872c3ec4d5705f53" is_pre_election: true
I20260927 11:08:29.621688 25082 raft_consensus.cc:2454] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ad61acf3e76459d93d1f114fdd47ab9 in term 0.
I20260927 11:08:29.621665 24946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64509167a1754d6c99162899e5ee5505" candidate_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44404d6e189f4c81a451dae98db9dac7" is_pre_election: true
I20260927 11:08:29.621729 24946 raft_consensus.cc:2454] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ad61acf3e76459d93d1f114fdd47ab9 in term 0.
I20260927 11:08:29.621873 24744 leader_election.cc:302] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ad61acf3e76459d93d1f114fdd47ab9, 44404d6e189f4c81a451dae98db9dac7; no voters:
I20260927 11:08:29.621987 25225 raft_consensus.cc:2781] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:29.622022 25225 raft_consensus.cc:482] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:29.622047 25225 raft_consensus.cc:3037] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:29.622571 25225 raft_consensus.cc:504] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.622696 25225 leader_election.cc:288] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [CANDIDATE]: Term 1 election: Requested vote from peers facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481), 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313)
I20260927 11:08:29.622815 25082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64509167a1754d6c99162899e5ee5505" candidate_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "facf2d136f2549d1872c3ec4d5705f53"
I20260927 11:08:29.622829 24946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64509167a1754d6c99162899e5ee5505" candidate_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44404d6e189f4c81a451dae98db9dac7"
I20260927 11:08:29.622879 25082 raft_consensus.cc:3037] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:29.622905 24946 raft_consensus.cc:3037] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:29.623617 24946 raft_consensus.cc:2454] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ad61acf3e76459d93d1f114fdd47ab9 in term 1.
I20260927 11:08:29.623617 25082 raft_consensus.cc:2454] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ad61acf3e76459d93d1f114fdd47ab9 in term 1.
I20260927 11:08:29.623831 24743 leader_election.cc:302] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ad61acf3e76459d93d1f114fdd47ab9, facf2d136f2549d1872c3ec4d5705f53; no voters:
I20260927 11:08:29.623945 25225 raft_consensus.cc:2781] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 FOLLOWER]: Leader election won for term 1
W20260927 11:08:29.624078 24859 tablet.cc:2369] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:29.624104 25225 raft_consensus.cc:686] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ad61acf3e76459d93d1f114fdd47ab9, State: Running, Role: LEADER
I20260927 11:08:29.624193 25225 consensus_queue.cc:237] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } }
I20260927 11:08:29.624567 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.624476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:46731)
0927 11:08:29.624510 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.624532 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 11:08:29.624835 24482 catalog_manager.cc:5495] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193). New cstate: current_term: 1 leader_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ad61acf3e76459d93d1f114fdd47ab9" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45643 } health_report { overall_health: HEALTHY } } }
I20260927 11:08:29.633304 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.630938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56928 (local address 127.17.95.193:45643)
0927 11:08:29.630980 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:29.630981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:29.630990 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:29.631044 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:29.631045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:29.631058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:29.631175 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:29.631311 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.631734 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.632526 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:29.632724 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:29.632808 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:29.632987 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:29.633086 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:29.633168 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:29.633214 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 11:08:29.633234 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 11:08:29.634176 24946 raft_consensus.cc:1261] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Refusing update from remote peer 3ad61acf3e76459d93d1f114fdd47ab9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:29.634248 25082 raft_consensus.cc:1261] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Refusing update from remote peer 3ad61acf3e76459d93d1f114fdd47ab9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:29.634538 25235 consensus_queue.cc:1035] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44404d6e189f4c81a451dae98db9dac7" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:29.634642 25225 consensus_queue.cc:1035] T 64509167a1754d6c99162899e5ee5505 P 3ad61acf3e76459d93d1f114fdd47ab9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "facf2d136f2549d1872c3ec4d5705f53" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:29.635115 25130 heartbeater.cc:503] Master 127.17.95.253:36361 was elected leader, sending a full tablet report...
I20260927 11:08:29.635274 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.635182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.195:44401)
0927 11:08:29.635221 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.635245 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 11:08:29.636056 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.635931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:48187)
0927 11:08:29.635997 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.636026 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 11:08:29.636710 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.636625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:49899)
0927 11:08:29.636657 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.636686 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 11:08:29.637145 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.636908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.194:40913)
0927 11:08:29.637088 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.637116 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260927 11:08:29.637208 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.637061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.193:42115)
0927 11:08:29.637160 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.637183 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260927 11:08:29.637475 25239 mvcc.cc:205] Tried to move back new op lower bound from 7333917940260020224 to 7333917940221050880. Current Snapshot: MvccSnapshot[applied={T|T < 7333917940260020224}]
I20260927 11:08:29.637913 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.637784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34341 (local address 127.17.95.195:51411)
0927 11:08:29.637863 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.637888 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:34341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 11:08:29.639384 24483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:29.639493 24483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:29.640534 24483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8d0490bd71446f48b3fee40f9f70383: Sending DeleteTablet for 3 replicas of tablet 64509167a1754d6c99162899e5ee5505
I20260927 11:08:29.640846 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet 64509167a1754d6c99162899e5ee5505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:29 UTC) from {username='oryx', principal='oryx/127.17.95.253@KRBTEST.COM'} at 127.0.0.1:58068
I20260927 11:08:29.640879 24790 tablet_service.cc:1514] Processing DeleteTablet for tablet 64509167a1754d6c99162899e5ee5505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:29 UTC) from {username='oryx', principal='oryx/127.17.95.253@KRBTEST.COM'} at 127.0.0.1:56916
I20260927 11:08:29.641014 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet 64509167a1754d6c99162899e5ee5505 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:29 UTC) from {username='oryx', principal='oryx/127.17.95.253@KRBTEST.COM'} at 127.0.0.1:40590
I20260927 11:08:29.641108 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24727
I20260927 11:08:29.641196 25250 tablet_replica.cc:330] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: stopping tablet replica
I20260927 11:08:29.641245 25250 raft_consensus.cc:2229] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:29.641283 25250 raft_consensus.cc:2258] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:29.642409 25248 tablet_replica.cc:330] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: stopping tablet replica
I20260927 11:08:29.642462 25248 raft_consensus.cc:2229] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:29.642503 25248 raft_consensus.cc:2258] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:29.642549 25250 ts_tablet_manager.cc:1905] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:29.642740 25248 ts_tablet_manager.cc:1905] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:29.644047 25250 ts_tablet_manager.cc:1918] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:29.644112 25250 log.cc:1196] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/64509167a1754d6c99162899e5ee5505
I20260927 11:08:29.644345 25250 ts_tablet_manager.cc:1939] T 64509167a1754d6c99162899e5ee5505 P facf2d136f2549d1872c3ec4d5705f53: Deleting consensus metadata
I20260927 11:08:29.644575 25248 ts_tablet_manager.cc:1918] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:29.644632 25248 log.cc:1196] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/64509167a1754d6c99162899e5ee5505
I20260927 11:08:29.644627 24467 catalog_manager.cc:4841] TS facf2d136f2549d1872c3ec4d5705f53 (127.17.95.195:45481): tablet 64509167a1754d6c99162899e5ee5505 (table test-table [id=94b741836a1b412792b6befcb2ca01ba]) successfully deleted
I20260927 11:08:29.644840 25248 ts_tablet_manager.cc:1939] T 64509167a1754d6c99162899e5ee5505 P 44404d6e189f4c81a451dae98db9dac7: Deleting consensus metadata
I20260927 11:08:29.645133 24468 catalog_manager.cc:4841] TS 44404d6e189f4c81a451dae98db9dac7 (127.17.95.194:34313): tablet 64509167a1754d6c99162899e5ee5505 (table test-table [id=94b741836a1b412792b6befcb2ca01ba]) successfully deleted
I20260927 11:08:29.647761 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:29.647674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45643 (local address 127.0.0.1:56934)
0927 11:08:29.647710 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:29.647733 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:45643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260927 11:08:29.647879 24468 catalog_manager.cc:4584] TS 3ad61acf3e76459d93d1f114fdd47ab9 (127.17.95.193:45643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64509167a1754d6c99162899e5ee5505: Network error: Client connection negotiation failed: client connection to 127.17.95.193:45643: connect: Connection refused (error 111)
I20260927 11:08:29.647909 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24863
I20260927 11:08:29.653550 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24999
I20260927 11:08:29.658967 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24348
I20260927 11:08:29.663702 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 24537
2026-09-27T11:08:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4569 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:29 dist-test-slave-6j0v krb5kdc[25253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:29 dist-test-slave-6j0v krb5kdc[25253](info): set up 2 sockets
Sep 27 11:08:29 dist-test-slave-6j0v krb5kdc[25253](info): commencing operation
krb5kdc: starting...
W20260927 11:08:30.683414 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:30 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab.
Entry for principal oryx/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab.
I20260927 11:08:30.793607 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33789
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab
--principal=oryx/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:33789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:30.870152 25266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:30.870334 25266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:30.870357 25266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:30.871452 25266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:30.871495 25266 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 11:08:30.871507 25266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:30.871519 25266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:30.871567 25266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:30.871582 25266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:30.873101 25266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:33789
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.254.keytab
--principal=oryx/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:30.873332 25266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:30.873567 25266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:30.878299 25271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:30.878324 25272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:30.878443 25266 server_base.cc:1034] running on GCE node
W20260927 11:08:30.878280 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:30.878711 25266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:30.878959 25266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:30.880126 25266 hybrid_clock.cc:648] HybridClock initialized: now 1790507310880119 us; error 39 us; skew 500 ppm
Sep 27 11:08:30 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507310, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:30.882558 25266 init.cc:377] Logged in from keytab as oryx/127.17.95.254@KRBTEST.COM (short username oryx)
I20260927 11:08:30.882966 25266 webserver.cc:460] Webserver started at http://127.17.95.254:42929/ using document root <none> and password file <none>
I20260927 11:08:30.883245 25266 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:30.883311 25266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:30.883450 25266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:30.884490 25266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "0f2cf97ee694430d8575be3bc896b3e7"
format_stamp: "Formatted at 2026-09-27 11:08:30 on dist-test-slave-6j0v"
server_key: "f8689c8b0dda0e4e897525fcf67cea6b"
server_key_iv: "7488e9c98753f806cc43aea5dce8fe9e"
server_key_version: "encryptionkey@0"
I20260927 11:08:30.884842 25266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "0f2cf97ee694430d8575be3bc896b3e7"
format_stamp: "Formatted at 2026-09-27 11:08:30 on dist-test-slave-6j0v"
server_key: "f8689c8b0dda0e4e897525fcf67cea6b"
server_key_iv: "7488e9c98753f806cc43aea5dce8fe9e"
server_key_version: "encryptionkey@0"
I20260927 11:08:30.886080 25266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:30.886763 25281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:30.886930 25266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:30.887020 25266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "0f2cf97ee694430d8575be3bc896b3e7"
format_stamp: "Formatted at 2026-09-27 11:08:30 on dist-test-slave-6j0v"
server_key: "f8689c8b0dda0e4e897525fcf67cea6b"
server_key_iv: "7488e9c98753f806cc43aea5dce8fe9e"
server_key_version: "encryptionkey@0"
I20260927 11:08:30.887135 25266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:30.898788 25266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:30.900501 25266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:30.900650 25266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:30.904739 25266 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:33789
I20260927 11:08:30.904838 25343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:33789 every 8 connection(s)
I20260927 11:08:30.905097 25266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:30.905764 25344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:30.907559 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 25266
I20260927 11:08:30.907657 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:30.907859 17791 external_mini_cluster.cc:1428] Setting key d242b6a127f02464a35f0fd6dc56c041
I20260927 11:08:30.908434 25344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Bootstrap starting.
I20260927 11:08:30.909003 25344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:30.909319 25344 log.cc:824] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:30.910125 25344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: No bootstrap required, opened a new log
I20260927 11:08:30.912417 25344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2cf97ee694430d8575be3bc896b3e7" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33789 } }
I20260927 11:08:30.912643 25344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:30.912681 25344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2cf97ee694430d8575be3bc896b3e7, State: Initialized, Role: FOLLOWER
I20260927 11:08:30.912828 25344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2cf97ee694430d8575be3bc896b3e7" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33789 } }
I20260927 11:08:30.912935 25344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:30.912985 25344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:30.913060 25344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:30.913606 25344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2cf97ee694430d8575be3bc896b3e7" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33789 } }
I20260927 11:08:30.913693 25344 leader_election.cc:302] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f2cf97ee694430d8575be3bc896b3e7; no voters:
I20260927 11:08:30.913893 25344 leader_election.cc:288] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:30.913942 25349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:30.914093 25349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [term 1 LEADER]: Becoming Leader. State: Replica: 0f2cf97ee694430d8575be3bc896b3e7, State: Running, Role: LEADER
I20260927 11:08:30.914187 25344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:30.914256 25349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2cf97ee694430d8575be3bc896b3e7" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33789 } }
I20260927 11:08:30.914602 25349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f2cf97ee694430d8575be3bc896b3e7. Latest consensus state: current_term: 1 leader_uuid: "0f2cf97ee694430d8575be3bc896b3e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2cf97ee694430d8575be3bc896b3e7" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33789 } } }
I20260927 11:08:30.914614 25351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f2cf97ee694430d8575be3bc896b3e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2cf97ee694430d8575be3bc896b3e7" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33789 } } }
I20260927 11:08:30.914690 25349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:30.914742 25351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:30.914979 25356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 27 11:08:30 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:30.915509 25356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:30.917469 25356 catalog_manager.cc:1347] Generated new cluster ID: 946a1d32678b4ff386d17be20ae79159
I20260927 11:08:30.917526 25356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:30.919409 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:30.908366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.17.95.254:33789)
0927 11:08:30.908582 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:30.908588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:30.908611 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:08:30.909019 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:30.909024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:30.909102 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:30.909218 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:30.910039 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:30.911445 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:30.912168 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:30.912775 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:30.912922 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:30.915800 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:30.915803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:30.915803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:30.915814 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:30.917896 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:30.918301 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:30.918304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:30.918305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:30.918409 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:30.918614 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:30.918615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:30.918616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:30.918836 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:30.919021 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:30.919121 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 11:08:30.919248 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260927 11:08:30.934998 25356 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:30.935727 25356 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:30.943316 25356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Generated new TSK 0
I20260927 11:08:30.943490 25356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab.
Entry for principal oryx/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab.
I20260927 11:08:30.974591 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33789
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:31.055197 25370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:31.055436 25370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:31.055475 25370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:31.056599 25370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:31.056685 25370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:31.056721 25370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:31.056766 25370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:31.058270 25370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:31.058542 25370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:31.058754 25370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:08:31.063663 25370 server_base.cc:1034] running on GCE node
W20260927 11:08:31.063627 25376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:31.063647 25378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:31.063627 25375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:31.064041 25370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:31.064277 25370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:31.065431 25370 hybrid_clock.cc:648] HybridClock initialized: now 1790507311065418 us; error 31 us; skew 500 ppm
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:31.067894 25370 init.cc:377] Logged in from keytab as oryx/127.17.95.193@KRBTEST.COM (short username oryx)
I20260927 11:08:31.068285 25370 webserver.cc:460] Webserver started at http://127.17.95.193:45909/ using document root <none> and password file <none>
I20260927 11:08:31.068511 25370 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:31.068573 25370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:31.068712 25370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:31.069701 25370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "60aba79e58933b596eca33b705137a43"
server_key_iv: "bf6bf91017bb85687826d0ff2534558f"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.070041 25370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "60aba79e58933b596eca33b705137a43"
server_key_iv: "bf6bf91017bb85687826d0ff2534558f"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.071319 25370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:31.072007 25385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.072170 25370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:31.072293 25370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "60aba79e58933b596eca33b705137a43"
server_key_iv: "bf6bf91017bb85687826d0ff2534558f"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.072362 25370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:31.080230 25370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:31.081964 25370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:31.082103 25370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:31.082530 25370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:31.082587 25370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.082628 25370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:31.082656 25370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:31.089823 25370 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:46775
I20260927 11:08:31.089888 25500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:46775 every 8 connection(s)
I20260927 11:08:31.090200 25370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:31.092200 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.083034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:43275 (local address 127.17.95.254:33789)
0927 11:08:31.083116 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.083119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.083304 (+ 185us) server_negotiation.cc:407] Connection header received
0927 11:08:31.083809 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.083811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.083824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.083891 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.084492 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.085290 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.086366 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.086717 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.086788 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.089482 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.089484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.089484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.089492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.091178 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.091540 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.091541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.091541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.091590 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.091768 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.091769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.091769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.091858 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.091898 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.092082 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.092143 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 11:08:31.092382 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.083019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.17.95.193:43275)
0927 11:08:31.083191 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.083223 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.083702 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.083966 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.083970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.084213 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.084379 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.084383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.085402 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.085403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.086262 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.086264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.086840 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.086841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.087016 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.087630 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.087641 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.089382 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.091242 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.091246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.091248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.091460 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.091644 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.091644 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.091645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.091722 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.091899 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.091901 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.092000 (+ 99us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.092067 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.092169 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260927 11:08:31.092834 25501 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33789
I20260927 11:08:31.092914 25501 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:31.093114 25501 heartbeater.cc:511] Master 127.17.95.254:33789 requested a full tablet report, sending...
I20260927 11:08:31.093647 25297 ts_manager.cc:194] Registered new tserver with Master: 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775)
I20260927 11:08:31.094528 25297 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:43275
I20260927 11:08:31.098668 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.093865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.17.95.193:32807)
0927 11:08:31.093997 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.094007 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.094061 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.094171 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.094172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.094342 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.094399 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.094400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.095217 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.095217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.095692 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.095693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.096016 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.096017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.096059 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.096383 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.096387 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.096632 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.097938 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.097938 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.097939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.098089 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.098257 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.098257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.098258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.098303 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.098469 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.098470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.098503 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.098546 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.098611 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260927 11:08:31.098670 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.093956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:32807 (local address 127.17.95.254:33789)
0927 11:08:31.093997 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.093999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.094057 (+ 58us) server_negotiation.cc:407] Connection header received
0927 11:08:31.094112 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.094112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.094121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.094168 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.094462 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.095133 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.095769 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.095937 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.096001 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.096732 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.096733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.096733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.096739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.097872 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.098151 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.098152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.098152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.098202 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.098343 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.098343 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.098344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.098425 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.098477 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.098562 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.098618 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 11:08:31.099308 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 25370
I20260927 11:08:31.099404 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:31.099644 17791 external_mini_cluster.cc:1428] Setting key 4a818db472b9117344e0199d2f395069
WARNING: no policy specified for oryx/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab.
Entry for principal oryx/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab.
I20260927 11:08:31.123566 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33789
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:31.203291 25510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:31.203464 25510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:31.203485 25510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:31.204618 25510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:31.204656 25510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:31.204684 25510 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:31.204700 25510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:31.206089 25510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:31.206301 25510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:31.206522 25510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:31.211256 25518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:31.211308 25515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:31.211413 25510 server_base.cc:1034] running on GCE node
W20260927 11:08:31.211236 25516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:31.211696 25510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:31.211951 25510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:31.213109 25510 hybrid_clock.cc:648] HybridClock initialized: now 1790507311213091 us; error 42 us; skew 500 ppm
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:31.215840 25510 init.cc:377] Logged in from keytab as oryx/127.17.95.194@KRBTEST.COM (short username oryx)
I20260927 11:08:31.216323 25510 webserver.cc:460] Webserver started at http://127.17.95.194:45125/ using document root <none> and password file <none>
I20260927 11:08:31.216543 25510 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:31.216586 25510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:31.216728 25510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:31.217733 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "365035a25b204076a930a763c8c23e4e"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "b9b3cf24c76aa0dda75be4725e8688b0"
server_key_iv: "e5e121a3d30fbcf1c40e0a9331a11585"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.218079 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "365035a25b204076a930a763c8c23e4e"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "b9b3cf24c76aa0dda75be4725e8688b0"
server_key_iv: "e5e121a3d30fbcf1c40e0a9331a11585"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.219322 25510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:31.219987 25525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.220158 25510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:31.220225 25510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "365035a25b204076a930a763c8c23e4e"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "b9b3cf24c76aa0dda75be4725e8688b0"
server_key_iv: "e5e121a3d30fbcf1c40e0a9331a11585"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.220307 25510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:31.234402 25510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:31.236174 25510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:31.236320 25510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:31.236783 25510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:31.236822 25510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.236842 25510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:31.236884 25510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:31.243863 25510 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:40297
I20260927 11:08:31.243942 25640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:40297 every 8 connection(s)
I20260927 11:08:31.244277 25510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:31.246308 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.237246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:42307 (local address 127.17.95.254:33789)
0927 11:08:31.237318 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.237320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.237543 (+ 223us) server_negotiation.cc:407] Connection header received
0927 11:08:31.238230 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.238231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.238240 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.238300 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.239199 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.239906 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.240848 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.241056 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.241137 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.243756 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.243757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.243758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.243765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.245224 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.245582 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.245583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.245583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.245630 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.245821 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.245822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.245822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.245917 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.245978 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.246194 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.246250 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 11:08:31.246470 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.237210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.17.95.194:42307)
0927 11:08:31.237408 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.237455 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.238129 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.238388 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.238393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.238772 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.239062 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.239091 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.240009 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.240010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.240759 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.240762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.241158 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.241159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.241274 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.241755 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.241765 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.243656 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.245291 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.245295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.245297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.245502 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.245684 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.245684 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.245684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.245770 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.245987 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.245991 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.246089 (+ 98us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.246162 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.246267 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260927 11:08:31.246963 25641 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33789
I20260927 11:08:31.247040 25641 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:31.247263 25641 heartbeater.cc:511] Master 127.17.95.254:33789 requested a full tablet report, sending...
I20260927 11:08:31.247589 25298 ts_manager.cc:194] Registered new tserver with Master: 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297)
I20260927 11:08:31.247960 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 25510
I20260927 11:08:31.248054 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:31.248126 25298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:42307
I20260927 11:08:31.248242 17791 external_mini_cluster.cc:1428] Setting key 9399e50eed408af78d71ce5874aca29a
I20260927 11:08:31.253449 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.248198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:42305 (local address 127.17.95.254:33789)
0927 11:08:31.248232 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.248234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.248242 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:31.248357 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.248357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.248367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.248415 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.248785 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.249453 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.250094 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.250293 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.250364 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.251167 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.251168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.251169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.251175 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.252362 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.252693 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.252694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.252694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.252747 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.252937 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.252938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.252938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.253072 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.253145 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.253301 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.253369 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:31.253458 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.248057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.17.95.194:42305)
0927 11:08:31.248160 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.248170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.248263 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.248509 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.248511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.248650 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.248700 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.248701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.249553 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.249554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.250010 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.250011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.250362 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.250363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.250425 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.250811 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.250815 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.251050 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.252446 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.252447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.252447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.252604 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.252814 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.252815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.252815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.252869 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.253138 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.253138 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.253188 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.253265 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.253358 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":55,"threads_started":1}
WARNING: no policy specified for oryx/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab.
Entry for principal oryx/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab.
I20260927 11:08:31.271564 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33789
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:31.348788 25650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:31.348980 25650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:31.349001 25650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:31.350095 25650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:31.350132 25650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:31.350152 25650 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 11:08:31.350168 25650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:31.351601 25650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.95.254:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:31.351830 25650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:31.352054 25650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:31.356822 25658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:31.356853 25655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:31.356819 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:31.356910 25650 server_base.cc:1034] running on GCE node
I20260927 11:08:31.357234 25650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:31.357414 25650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:31.358541 25650 hybrid_clock.cc:648] HybridClock initialized: now 1790507311358532 us; error 26 us; skew 500 ppm
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:31.365559 25650 init.cc:377] Logged in from keytab as oryx/127.17.95.195@KRBTEST.COM (short username oryx)
I20260927 11:08:31.366039 25650 webserver.cc:460] Webserver started at http://127.17.95.195:35147/ using document root <none> and password file <none>
I20260927 11:08:31.366338 25650 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:31.366401 25650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:31.366536 25650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:31.367846 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "1b15a03c6ef34df290c6c09745cdf34f"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "d329938b45ce15f1b80ab4f883291a67"
server_key_iv: "d54151ab38a564a03e080e61826d06bc"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.368211 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "1b15a03c6ef34df290c6c09745cdf34f"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "d329938b45ce15f1b80ab4f883291a67"
server_key_iv: "d54151ab38a564a03e080e61826d06bc"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.369937 25650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:31.370571 25665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.370757 25650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:31.370875 25650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "1b15a03c6ef34df290c6c09745cdf34f"
format_stamp: "Formatted at 2026-09-27 11:08:31 on dist-test-slave-6j0v"
server_key: "d329938b45ce15f1b80ab4f883291a67"
server_key_iv: "d54151ab38a564a03e080e61826d06bc"
server_key_version: "encryptionkey@0"
I20260927 11:08:31.370960 25650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:31.385213 25650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:31.387346 25650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:31.387516 25650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:31.387993 25650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:31.388054 25650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.388087 25650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:31.388116 25650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.394783 25650 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:37437
I20260927 11:08:31.394848 25780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:37437 every 8 connection(s)
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:31.395187 25650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:31.395839 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 25650
I20260927 11:08:31.395961 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:31.396157 17791 external_mini_cluster.cc:1428] Setting key f903b9a16fe43fdb92209ed2a903304d
I20260927 11:08:31.398458 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.388444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:59607 (local address 127.17.95.254:33789)
0927 11:08:31.388516 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.388518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.388686 (+ 168us) server_negotiation.cc:407] Connection header received
0927 11:08:31.389190 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.389192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.389206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.389282 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.390149 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.390952 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.392473 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.392804 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.392875 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.395504 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.395505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.395506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.395514 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.397129 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.397519 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.397521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.397521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.397578 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.397834 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.397835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.397835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.397949 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.398016 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.398285 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.398374 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 11:08:31.398618 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.388418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.17.95.195:59607)
0927 11:08:31.388550 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.388588 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.389085 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.389356 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.389360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.389770 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.390041 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.390048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.391060 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.391061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.392365 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.392367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.392880 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.392881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.393025 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.393610 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.393621 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.395386 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.397210 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.397216 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.397218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.397439 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.397651 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.397652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.397653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.397762 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.398018 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.398024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.398155 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.398254 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.398409 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260927 11:08:31.399063 25781 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33789
I20260927 11:08:31.399158 25781 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:31.399328 25781 heartbeater.cc:511] Master 127.17.95.254:33789 requested a full tablet report, sending...
I20260927 11:08:31.399682 25298 ts_manager.cc:194] Registered new tserver with Master: 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:31.400144 25298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:59607
I20260927 11:08:31.405032 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.400089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:53855 (local address 127.17.95.254:33789)
0927 11:08:31.400127 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.400129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.400258 (+ 129us) server_negotiation.cc:407] Connection header received
0927 11:08:31.400285 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.400286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.400296 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.400364 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.400723 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.401260 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.401968 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.402164 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.402232 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.403059 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.403059 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.403060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.403064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.404276 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.404561 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.404562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.404562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.404606 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.404738 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.404738 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.404738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.404814 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.404855 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.404933 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.404981 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:08:31.405050 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.400035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.17.95.195:53855)
0927 11:08:31.400177 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.400186 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.400247 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.400368 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.400369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.400573 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.400626 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.400627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.401350 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.401350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.401898 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.401899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.402242 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.402243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.402285 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.402691 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.402696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.402967 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.404353 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.404354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.404354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.404501 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.404656 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.404657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.404657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.404703 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.404846 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.404846 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.404875 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.404918 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.404989 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260927 11:08:31.407841 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 11:08:31.427896 25347 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:31.422689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.17.95.254:33789)
0927 11:08:31.422748 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.422751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.422760 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:31.422807 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.422810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.422823 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.422863 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.423452 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.424030 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.424782 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.424960 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.425022 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.427796 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36680: BlockingRecv error: recv got EOF from 127.0.0.1:36680 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507311, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:31.434842 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.428714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.17.95.254:33789)
0927 11:08:31.428766 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.428768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.428793 (+ 25us) server_negotiation.cc:407] Connection header received
0927 11:08:31.428860 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.428860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.428868 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.428925 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.429315 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.429765 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.430368 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.430539 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.430585 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.432563 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.432564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.432564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.432570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.433949 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.434263 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.434264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.434264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.434328 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.434491 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.434492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.434492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.434575 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.434641 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.434721 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.434786 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:31.436282 25298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:31.437789 25298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:31.444161 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.439834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:40297 (local address 127.0.0.1:49850)
0927 11:08:31.439982 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.440020 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.440076 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.440409 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.440412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.440413 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.440475 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.440477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.441676 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.441676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.442932 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.442935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.443907 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.443907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.443960 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.443971 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.444010 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.444045 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":32,"thread_start_us":55,"threads_started":1}
I20260927 11:08:31.444202 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.439932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.17.95.194:40297)
0927 11:08:31.440224 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.440230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.440241 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:31.440281 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.440283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.440297 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.440379 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.440533 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.441589 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.443030 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.443819 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.443904 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.443995 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.444046 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.444094 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":91,"threads_started":1}
I20260927 11:08:31.444307 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.440006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:46775 (local address 127.0.0.1:34772)
0927 11:08:31.440493 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.440502 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.440549 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.440726 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.440726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.440728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.440775 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.440776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.441928 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.441929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.443134 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.443144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.444094 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.444094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.444145 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.444149 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.444187 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.444238 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":45,"thread_start_us":42,"threads_started":1}
I20260927 11:08:31.444732 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.440193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.17.95.193:46775)
0927 11:08:31.440363 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.440368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.440608 (+ 240us) server_negotiation.cc:407] Connection header received
0927 11:08:31.440648 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.440650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.440664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.440729 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.440851 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.441832 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.443219 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.444014 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.444094 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.444168 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.444621 (+ 453us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.444658 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260927 11:08:31.444826 25575 tablet_service.cc:1467] Processing CreateTablet for tablet e717e4a29d254d9db34c4aa9d49b2001 (DEFAULT_TABLE table=test-table [id=9f10512c30bb46149741854ca2306f65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:31.445227 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e717e4a29d254d9db34c4aa9d49b2001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.445386 25435 tablet_service.cc:1467] Processing CreateTablet for tablet e717e4a29d254d9db34c4aa9d49b2001 (DEFAULT_TABLE table=test-table [id=9f10512c30bb46149741854ca2306f65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:31.445686 25435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e717e4a29d254d9db34c4aa9d49b2001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.445909 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.439834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:37437 (local address 127.0.0.1:33530)
0927 11:08:31.440085 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.440093 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.440127 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.440988 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.440988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.440990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.441049 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.441051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.441902 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.441903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.443134 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.443137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.445632 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.445633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.445697 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.445701 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.445783 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.445823 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":38,"thread_start_us":51,"threads_started":1}
I20260927 11:08:31.445941 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.439932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.17.95.195:37437)
0927 11:08:31.440848 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.440852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.440860 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:31.440893 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.440895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.440914 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.440989 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.441108 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.441816 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.444854 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.445525 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.445626 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.445697 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.445796 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.445831 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":93,"threads_started":1}
I20260927 11:08:31.449308 25806 tablet_bootstrap.cc:492] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Bootstrap starting.
I20260927 11:08:31.449782 25807 tablet_bootstrap.cc:492] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Bootstrap starting.
I20260927 11:08:31.449843 25715 tablet_service.cc:1467] Processing CreateTablet for tablet e717e4a29d254d9db34c4aa9d49b2001 (DEFAULT_TABLE table=test-table [id=9f10512c30bb46149741854ca2306f65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:31.449986 25806 tablet_bootstrap.cc:654] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.450189 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e717e4a29d254d9db34c4aa9d49b2001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.450299 25806 log.cc:824] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:31.450495 25807 tablet_bootstrap.cc:654] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.450856 25807 log.cc:824] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:31.451023 25806 tablet_bootstrap.cc:492] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: No bootstrap required, opened a new log
I20260927 11:08:31.451126 25806 ts_tablet_manager.cc:1397] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:31.451735 25807 tablet_bootstrap.cc:492] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: No bootstrap required, opened a new log
I20260927 11:08:31.451858 25807 ts_tablet_manager.cc:1397] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:31.452749 25806 raft_consensus.cc:348] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.452878 25806 raft_consensus.cc:374] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.452903 25806 raft_consensus.cc:729] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.453028 25806 consensus_queue.cc:260] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.453306 25806 ts_tablet_manager.cc:1428] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:31.453323 25641 heartbeater.cc:503] Master 127.17.95.254:33789 was elected leader, sending a full tablet report...
I20260927 11:08:31.453732 25811 tablet_bootstrap.cc:492] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Bootstrap starting.
I20260927 11:08:31.454176 25807 raft_consensus.cc:348] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.454325 25807 raft_consensus.cc:374] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.454327 25811 tablet_bootstrap.cc:654] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.454365 25807 raft_consensus.cc:729] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7816405ea2f34b91bf7b32da8e9e8e46, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.454475 25807 consensus_queue.cc:260] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.454627 25811 log.cc:824] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:31.454725 25807 ts_tablet_manager.cc:1428] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 11:08:31.454797 25501 heartbeater.cc:503] Master 127.17.95.254:33789 was elected leader, sending a full tablet report...
I20260927 11:08:31.455328 25811 tablet_bootstrap.cc:492] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: No bootstrap required, opened a new log
I20260927 11:08:31.455412 25811 ts_tablet_manager.cc:1397] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:31.456777 25811 raft_consensus.cc:348] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.456916 25811 raft_consensus.cc:374] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.456960 25811 raft_consensus.cc:729] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b15a03c6ef34df290c6c09745cdf34f, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.457063 25811 consensus_queue.cc:260] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.457304 25811 ts_tablet_manager.cc:1428] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:31.457357 25781 heartbeater.cc:503] Master 127.17.95.254:33789 was elected leader, sending a full tablet report...
W20260927 11:08:31.495105 25642 tablet.cc:2369] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:31.557686 25814 raft_consensus.cc:482] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:31.557793 25814 raft_consensus.cc:504] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.558120 25814 leader_election.cc:288] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297)
I20260927 11:08:31.560889 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.558357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:43093 (local address 127.17.95.193:46775)
0927 11:08:31.558407 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.558409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.558558 (+ 149us) server_negotiation.cc:407] Connection header received
0927 11:08:31.558620 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.558621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.558630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.558708 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.558849 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.559315 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.560174 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.560536 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.560612 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.560620 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.560792 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.560825 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:31.560892 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.558360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:41755 (local address 127.17.95.194:40297)
0927 11:08:31.558401 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.558403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.558412 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:31.558523 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.558524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.558532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.558600 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.558767 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.559262 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.560169 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.560525 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.560599 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.560608 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.560792 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.560827 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:08:31.560914 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.558313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:46775 (local address 127.17.95.195:43093)
0927 11:08:31.558468 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.558478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.558567 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.558709 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.558709 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.558711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.558788 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.558789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.559396 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.559397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.560089 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.560091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.560611 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.560612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.560683 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.560686 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.560784 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.560823 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":20,"thread_start_us":87,"threads_started":1}
I20260927 11:08:31.560923 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.558307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:40297 (local address 127.17.95.195:41755)
0927 11:08:31.558363 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.558374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.558418 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.558619 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.558620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.558622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.558689 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.558690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.559331 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.559332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.560075 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.560077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.560595 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.560595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.560682 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.560685 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.560785 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.560826 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 11:08:31.561453 25595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e717e4a29d254d9db34c4aa9d49b2001" candidate_uuid: "1b15a03c6ef34df290c6c09745cdf34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "365035a25b204076a930a763c8c23e4e" is_pre_election: true
I20260927 11:08:31.561491 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e717e4a29d254d9db34c4aa9d49b2001" candidate_uuid: "1b15a03c6ef34df290c6c09745cdf34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" is_pre_election: true
I20260927 11:08:31.561606 25595 raft_consensus.cc:2454] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b15a03c6ef34df290c6c09745cdf34f in term 0.
I20260927 11:08:31.561625 25455 raft_consensus.cc:2454] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b15a03c6ef34df290c6c09745cdf34f in term 0.
I20260927 11:08:31.561836 25666 leader_election.cc:302] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 365035a25b204076a930a763c8c23e4e; no voters:
I20260927 11:08:31.561995 25814 raft_consensus.cc:2781] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:31.562031 25814 raft_consensus.cc:482] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:31.562045 25814 raft_consensus.cc:3037] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:31.562863 25814 raft_consensus.cc:504] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.562994 25814 leader_election.cc:288] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [CANDIDATE]: Term 1 election: Requested vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297)
I20260927 11:08:31.563133 25595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e717e4a29d254d9db34c4aa9d49b2001" candidate_uuid: "1b15a03c6ef34df290c6c09745cdf34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "365035a25b204076a930a763c8c23e4e"
I20260927 11:08:31.563130 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e717e4a29d254d9db34c4aa9d49b2001" candidate_uuid: "1b15a03c6ef34df290c6c09745cdf34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
I20260927 11:08:31.563194 25595 raft_consensus.cc:3037] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:31.563195 25455 raft_consensus.cc:3037] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:31.564014 25455 raft_consensus.cc:2454] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b15a03c6ef34df290c6c09745cdf34f in term 1.
I20260927 11:08:31.564014 25595 raft_consensus.cc:2454] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b15a03c6ef34df290c6c09745cdf34f in term 1.
I20260927 11:08:31.564170 25666 leader_election.cc:302] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 365035a25b204076a930a763c8c23e4e; no voters:
I20260927 11:08:31.564260 25814 raft_consensus.cc:2781] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:31.564383 25814 raft_consensus.cc:686] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 LEADER]: Becoming Leader. State: Replica: 1b15a03c6ef34df290c6c09745cdf34f, State: Running, Role: LEADER
I20260927 11:08:31.564504 25814 consensus_queue.cc:237] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } }
I20260927 11:08:31.565274 25297 catalog_manager.cc:5495] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195). New cstate: current_term: 1 leader_uuid: "1b15a03c6ef34df290c6c09745cdf34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:31.587893 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.585435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.17.95.195:37437)
0927 11:08:31.585497 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.585499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.585508 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:31.585591 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.585592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.585618 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.585686 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:31.585842 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.586282 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.587048 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.587282 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.587385 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.587561 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:31.587663 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:31.587746 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.587798 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.587817 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:08:31.589730 25455 raft_consensus.cc:1261] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Refusing update from remote peer 1b15a03c6ef34df290c6c09745cdf34f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:31.589740 25595 raft_consensus.cc:1261] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Refusing update from remote peer 1b15a03c6ef34df290c6c09745cdf34f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:31.590034 25816 consensus_queue.cc:1035] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:31.590230 25814 consensus_queue.cc:1035] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [LEADER]: Connected to new peer: Peer: permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 11:08:31.590847 25502 tablet.cc:2369] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:31.592667 25821 mvcc.cc:205] Tried to move back new op lower bound from 7333917948268560384 to 7333917948168900608. Current Snapshot: MvccSnapshot[applied={T|T < 7333917948268560384}]
I20260927 11:08:31.593096 25820 mvcc.cc:205] Tried to move back new op lower bound from 7333917948268560384 to 7333917948168900608. Current Snapshot: MvccSnapshot[applied={T|T < 7333917948268560384}]
I20260927 11:08:31.594794 25298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:31.594877 25298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:31.595734 25298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Sending DeleteTablet for 3 replicas of tablet e717e4a29d254d9db34c4aa9d49b2001
I20260927 11:08:31.596086 25435 tablet_service.cc:1514] Processing DeleteTablet for tablet e717e4a29d254d9db34c4aa9d49b2001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:31 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260927 11:08:31.596287 25715 tablet_service.cc:1514] Processing DeleteTablet for tablet e717e4a29d254d9db34c4aa9d49b2001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:31 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33530
I20260927 11:08:31.596338 25575 tablet_service.cc:1514] Processing DeleteTablet for tablet e717e4a29d254d9db34c4aa9d49b2001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:31 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49850
I20260927 11:08:31.596426 25298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 11:08:31.596549 25829 tablet_replica.cc:330] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: stopping tablet replica
W20260927 11:08:31.596580 25298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:31.596652 25829 raft_consensus.cc:2229] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:31.596747 25829 raft_consensus.cc:2258] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:31.597177 25829 ts_tablet_manager.cc:1905] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:31.597697 25435 tablet_service.cc:1467] Processing CreateTablet for tablet 49844795c0e5488c8ae82f938d201e7b (DEFAULT_TABLE table=test-table [id=754d2da7d22547b88d1bf69b57a94c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:31.597723 25715 tablet_service.cc:1467] Processing CreateTablet for tablet 49844795c0e5488c8ae82f938d201e7b (DEFAULT_TABLE table=test-table [id=754d2da7d22547b88d1bf69b57a94c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:31.597815 25435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49844795c0e5488c8ae82f938d201e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.597831 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49844795c0e5488c8ae82f938d201e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.599048 25829 ts_tablet_manager.cc:1918] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:31.599115 25807 tablet_bootstrap.cc:492] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Bootstrap starting.
I20260927 11:08:31.599402 25575 tablet_service.cc:1467] Processing CreateTablet for tablet 49844795c0e5488c8ae82f938d201e7b (DEFAULT_TABLE table=test-table [id=754d2da7d22547b88d1bf69b57a94c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:31.599510 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49844795c0e5488c8ae82f938d201e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.599774 25807 tablet_bootstrap.cc:654] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.599860 25811 tablet_bootstrap.cc:492] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Bootstrap starting.
I20260927 11:08:31.599134 25829 log.cc:1196] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/e717e4a29d254d9db34c4aa9d49b2001
I20260927 11:08:31.600271 25811 tablet_bootstrap.cc:654] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.600359 25829 ts_tablet_manager.cc:1939] T e717e4a29d254d9db34c4aa9d49b2001 P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting consensus metadata
I20260927 11:08:31.600752 25806 tablet_bootstrap.cc:492] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Bootstrap starting.
I20260927 11:08:31.600811 25284 catalog_manager.cc:4841] TS 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775): tablet e717e4a29d254d9db34c4aa9d49b2001 (table test-table [id=9f10512c30bb46149741854ca2306f65]) successfully deleted
I20260927 11:08:31.601152 25806 tablet_bootstrap.cc:654] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.601501 25830 tablet_replica.cc:330] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: stopping tablet replica
I20260927 11:08:31.601603 25811 tablet_bootstrap.cc:492] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: No bootstrap required, opened a new log
I20260927 11:08:31.601661 25811 ts_tablet_manager.cc:1397] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:31.601755 25807 tablet_bootstrap.cc:492] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: No bootstrap required, opened a new log
I20260927 11:08:31.601799 25807 ts_tablet_manager.cc:1397] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 11:08:31.601814 25811 raft_consensus.cc:348] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.601889 25811 raft_consensus.cc:374] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.601913 25811 raft_consensus.cc:729] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b15a03c6ef34df290c6c09745cdf34f, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.601913 25807 raft_consensus.cc:348] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.601979 25807 raft_consensus.cc:374] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.601958 25811 consensus_queue.cc:260] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.602005 25807 raft_consensus.cc:729] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7816405ea2f34b91bf7b32da8e9e8e46, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.602044 25811 ts_tablet_manager.cc:1428] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.602041 25807 consensus_queue.cc:260] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.602120 25807 ts_tablet_manager.cc:1428] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.602191 25831 tablet_replica.cc:330] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: stopping tablet replica
I20260927 11:08:31.602283 25831 raft_consensus.cc:2229] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:31.602391 25831 raft_consensus.cc:2258] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:31.601594 25830 raft_consensus.cc:2229] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:31.602597 25830 raft_consensus.cc:2258] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:31.603129 25806 tablet_bootstrap.cc:492] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: No bootstrap required, opened a new log
I20260927 11:08:31.603128 25830 ts_tablet_manager.cc:1905] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:31.603180 25806 ts_tablet_manager.cc:1397] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:31.603327 25806 raft_consensus.cc:348] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.603413 25806 raft_consensus.cc:374] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.603440 25806 raft_consensus.cc:729] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.603443 25831 ts_tablet_manager.cc:1905] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:31.603494 25806 consensus_queue.cc:260] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.603575 25806 ts_tablet_manager.cc:1428] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.604723 25831 ts_tablet_manager.cc:1918] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:31.604803 25831 log.cc:1196] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/e717e4a29d254d9db34c4aa9d49b2001
I20260927 11:08:31.604947 25830 ts_tablet_manager.cc:1918] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:31.605047 25830 log.cc:1196] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/e717e4a29d254d9db34c4aa9d49b2001
I20260927 11:08:31.605070 25831 ts_tablet_manager.cc:1939] T e717e4a29d254d9db34c4aa9d49b2001 P 365035a25b204076a930a763c8c23e4e: Deleting consensus metadata
I20260927 11:08:31.605333 25830 ts_tablet_manager.cc:1939] T e717e4a29d254d9db34c4aa9d49b2001 P 1b15a03c6ef34df290c6c09745cdf34f: Deleting consensus metadata
I20260927 11:08:31.605513 25282 catalog_manager.cc:4841] TS 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297): tablet e717e4a29d254d9db34c4aa9d49b2001 (table test-table [id=9f10512c30bb46149741854ca2306f65]) successfully deleted
I20260927 11:08:31.605741 25284 catalog_manager.cc:4841] TS 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437): tablet e717e4a29d254d9db34c4aa9d49b2001 (table test-table [id=9f10512c30bb46149741854ca2306f65]) successfully deleted
W20260927 11:08:31.645696 25782 tablet.cc:2369] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:31.744360 25810 raft_consensus.cc:482] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:31.744503 25810 raft_consensus.cc:504] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.744879 25810 leader_election.cc:288] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:31.748026 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.745106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:38021 (local address 127.17.95.193:46775)
0927 11:08:31.745226 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.745229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.745238 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:31.745267 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.745268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.745278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.745316 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.745670 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.746150 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.747224 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.747632 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.747718 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.747728 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.747887 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.747940 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260927 11:08:31.748039 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.745029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:46775 (local address 127.17.95.194:38021)
0927 11:08:31.745083 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.745097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.745191 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.745508 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.745509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.745511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.745587 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.745589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.746237 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.746238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.747130 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.747132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.747718 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.747719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.747818 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.747822 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.747879 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.747920 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 11:08:31.748080 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.745248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:50999 (local address 127.17.95.195:37437)
0927 11:08:31.745301 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.745303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.745364 (+ 61us) server_negotiation.cc:407] Connection header received
0927 11:08:31.745425 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.745426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.745433 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.745498 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:31.745706 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.746168 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.747289 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.747707 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.747814 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.747823 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.747970 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.748010 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:31.748134 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.745120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:37437 (local address 127.17.95.194:50999)
0927 11:08:31.745302 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.745312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.745365 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.745538 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.745538 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.745540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:31.745611 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.745613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.746252 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.746253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.747184 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.747187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.747815 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.747816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.747889 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.747894 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.747948 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.747985 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":62,"threads_started":1}
I20260927 11:08:31.748287 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49844795c0e5488c8ae82f938d201e7b" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" is_pre_election: true
I20260927 11:08:31.748384 25455 raft_consensus.cc:2454] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 0.
I20260927 11:08:31.748377 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49844795c0e5488c8ae82f938d201e7b" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f" is_pre_election: true
I20260927 11:08:31.748509 25735 raft_consensus.cc:2454] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 0.
I20260927 11:08:31.748679 25528 leader_election.cc:302] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 365035a25b204076a930a763c8c23e4e; no voters:
I20260927 11:08:31.748801 25810 raft_consensus.cc:2781] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:31.748847 25810 raft_consensus.cc:482] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:31.748873 25810 raft_consensus.cc:3037] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:31.749562 25810 raft_consensus.cc:504] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.749698 25810 leader_election.cc:288] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 election: Requested vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:31.749887 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49844795c0e5488c8ae82f938d201e7b" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
I20260927 11:08:31.749917 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49844795c0e5488c8ae82f938d201e7b" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f"
I20260927 11:08:31.749970 25455 raft_consensus.cc:3037] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:31.749987 25735 raft_consensus.cc:3037] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:31.750699 25455 raft_consensus.cc:2454] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 1.
I20260927 11:08:31.750699 25735 raft_consensus.cc:2454] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 1.
I20260927 11:08:31.750886 25528 leader_election.cc:302] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 365035a25b204076a930a763c8c23e4e; no voters:
I20260927 11:08:31.750983 25810 raft_consensus.cc:2781] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:31.751112 25810 raft_consensus.cc:686] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 1 LEADER]: Becoming Leader. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Running, Role: LEADER
I20260927 11:08:31.751179 25810 consensus_queue.cc:237] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.751748 25297 catalog_manager.cc:5495] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 365035a25b204076a930a763c8c23e4e (127.17.95.194). New cstate: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } health_report { overall_health: UNKNOWN } } }
Sep 27 11:08:31 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507310, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.254@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:31.806695 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.799766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33789 (local address 127.0.0.1:36700)
0927 11:08:31.799928 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:31.799938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:31.800041 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:31.800178 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:31.800179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:31.800508 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:31.800560 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.800562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.801208 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.801209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.801733 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:31.801735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.802638 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.802639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:31.802680 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.803256 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:31.803260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:31.804540 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:31.805783 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.805788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.805790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.805973 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.806172 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:31.806173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:31.806173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:31.806226 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:31.806437 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:31.806442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:31.806481 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 11:08:31.806540 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 11:08:31.806615 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260927 11:08:31.806778 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:31.799826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.17.95.254:33789)
0927 11:08:31.799899 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:31.799901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:31.800014 (+ 113us) server_negotiation.cc:407] Connection header received
0927 11:08:31.800095 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:31.800096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:31.800106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:31.800175 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:31.800658 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.801134 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.801818 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:31.801990 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:31.802065 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:31.804614 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:31.804615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:31.804616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:31.804622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:31.805716 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.806053 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.806054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.806055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.806109 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:31.806286 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:31.806287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:31.806287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:31.806384 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:31.806432 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:31.806571 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 11:08:31.806681 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:08:31.808791 25297 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:36700:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 11:08:31.809006 25297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:31.810434 25715 tablet_service.cc:1467] Processing CreateTablet for tablet 642872ef75614369a536e3afe52216e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bbb89eb97226403392dfa4067624f82a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:31.810422 25435 tablet_service.cc:1467] Processing CreateTablet for tablet 642872ef75614369a536e3afe52216e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bbb89eb97226403392dfa4067624f82a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:31.810600 25435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642872ef75614369a536e3afe52216e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.810601 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642872ef75614369a536e3afe52216e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.811205 25575 tablet_service.cc:1467] Processing CreateTablet for tablet 642872ef75614369a536e3afe52216e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bbb89eb97226403392dfa4067624f82a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 11:08:31.811347 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642872ef75614369a536e3afe52216e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:31.812194 25811 tablet_bootstrap.cc:492] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: Bootstrap starting.
I20260927 11:08:31.812523 25811 tablet_bootstrap.cc:654] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.812517 25807 tablet_bootstrap.cc:492] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: Bootstrap starting.
I20260927 11:08:31.812822 25806 tablet_bootstrap.cc:492] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: Bootstrap starting.
I20260927 11:08:31.812980 25807 tablet_bootstrap.cc:654] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.813205 25806 tablet_bootstrap.cc:654] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:31.813668 25807 tablet_bootstrap.cc:492] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: No bootstrap required, opened a new log
I20260927 11:08:31.813722 25807 ts_tablet_manager.cc:1397] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:31.813854 25811 tablet_bootstrap.cc:492] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: No bootstrap required, opened a new log
I20260927 11:08:31.813895 25811 ts_tablet_manager.cc:1397] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:31.813864 25807 raft_consensus.cc:348] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.813916 25806 tablet_bootstrap.cc:492] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: No bootstrap required, opened a new log
I20260927 11:08:31.813946 25806 ts_tablet_manager.cc:1397] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:31.813932 25807 raft_consensus.cc:374] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.813969 25807 raft_consensus.cc:729] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7816405ea2f34b91bf7b32da8e9e8e46, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.814000 25811 raft_consensus.cc:348] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.814019 25807 consensus_queue.cc:260] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.814062 25811 raft_consensus.cc:374] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.814085 25811 raft_consensus.cc:729] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b15a03c6ef34df290c6c09745cdf34f, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.814141 25807 ts_tablet_manager.cc:1428] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:31.814080 25806 raft_consensus.cc:348] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.814169 25806 raft_consensus.cc:374] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:31.814191 25806 raft_consensus.cc:729] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Initialized, Role: FOLLOWER
I20260927 11:08:31.814162 25811 consensus_queue.cc:260] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.814226 25806 consensus_queue.cc:260] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:31.814260 25811 ts_tablet_manager.cc:1428] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:31.814252 25812 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:31.814318 25806 ts_tablet_manager.cc:1428] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:31.814330 25812 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:31.814388 25825 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:31.814471 25825 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:31.814429 25810 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:31.814514 25810 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:32.004534 25810 raft_consensus.cc:482] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:32.004630 25810 raft_consensus.cc:504] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.004772 25810 leader_election.cc:288] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:32.005054 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642872ef75614369a536e3afe52216e3" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" is_pre_election: true
I20260927 11:08:32.005079 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642872ef75614369a536e3afe52216e3" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f" is_pre_election: true
I20260927 11:08:32.005177 25455 raft_consensus.cc:2454] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 0.
I20260927 11:08:32.005172 25735 raft_consensus.cc:2454] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 0.
I20260927 11:08:32.005347 25528 leader_election.cc:302] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365035a25b204076a930a763c8c23e4e, 7816405ea2f34b91bf7b32da8e9e8e46; no voters:
I20260927 11:08:32.005448 25810 raft_consensus.cc:2781] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:32.005501 25810 raft_consensus.cc:482] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:32.005527 25810 raft_consensus.cc:3037] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.006171 25810 raft_consensus.cc:504] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.006295 25810 leader_election.cc:288] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 election: Requested vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:32.006487 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642872ef75614369a536e3afe52216e3" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f"
I20260927 11:08:32.006476 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642872ef75614369a536e3afe52216e3" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
I20260927 11:08:32.006563 25735 raft_consensus.cc:3037] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.006564 25455 raft_consensus.cc:3037] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.007380 25455 raft_consensus.cc:2454] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 1.
I20260927 11:08:32.007380 25735 raft_consensus.cc:2454] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 1.
I20260927 11:08:32.007601 25528 leader_election.cc:302] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 365035a25b204076a930a763c8c23e4e; no voters:
I20260927 11:08:32.007727 25810 raft_consensus.cc:2781] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:32.007820 25810 raft_consensus.cc:686] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [term 1 LEADER]: Becoming Leader. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Running, Role: LEADER
I20260927 11:08:32.007872 25810 consensus_queue.cc:237] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.008183 25836 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: TxnStatusTablet state changed. Reason: New leader 365035a25b204076a930a763c8c23e4e. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.008288 25836 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:32.008482 25848 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 11:08:32.008577 25848 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 11:08:32.008528 25298 catalog_manager.cc:5495] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 365035a25b204076a930a763c8c23e4e (127.17.95.194). New cstate: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:32.040853 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.037811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:40297 (local address 127.0.0.1:49852)
0927 11:08:32.037888 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.037902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:32.037961 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:32.038126 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:32.038127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:32.038128 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:32.038174 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.038176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.039138 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.039139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.039739 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.039741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.040208 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.040209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.040255 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.040268 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.040622 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.040631 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 11:08:32.040710 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:08:32.040759 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 11:08:32.040905 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.037884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49852 (local address 127.17.95.194:40297)
0927 11:08:32.037933 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.037935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.037946 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:32.038028 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.038029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.038049 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.038117 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.038240 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.038993 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.039821 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.040130 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.040202 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.040384 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.040519 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.040602 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.040762 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.040785 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:08:32.042028 25455 raft_consensus.cc:1261] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Refusing update from remote peer 365035a25b204076a930a763c8c23e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.042119 25735 raft_consensus.cc:1261] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Refusing update from remote peer 365035a25b204076a930a763c8c23e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.042351 25836 consensus_queue.cc:1035] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.042487 25810 consensus_queue.cc:1035] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.043061 25812 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: TxnStatusTablet state changed. Reason: New leader 365035a25b204076a930a763c8c23e4e. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.043113 25825 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: TxnStatusTablet state changed. Reason: New leader 365035a25b204076a930a763c8c23e4e. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.043181 25812 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:32.043200 25825 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:32.043851 25810 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.043928 25810 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:32.044011 25810 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.044059 25810 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 365035a25b204076a930a763c8c23e4e: This TxnStatusTablet replica's current role is: LEADER
I20260927 11:08:32.044243 25812 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.044389 25812 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 7816405ea2f34b91bf7b32da8e9e8e46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:32.045333 25819 mvcc.cc:205] Tried to move back new op lower bound from 7333917950121906176 to 7333917949984583680. Current Snapshot: MvccSnapshot[applied={T|T < 7333917950121906176}]
I20260927 11:08:32.045693 25825 tablet_replica.cc:439] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } } }
I20260927 11:08:32.045776 25825 tablet_replica.cc:442] T 642872ef75614369a536e3afe52216e3 P 1b15a03c6ef34df290c6c09745cdf34f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 11:08:32.048285 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.045599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.17.95.194:40297)
0927 11:08:32.045633 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.045634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.045642 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:32.045661 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.045662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.045671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.045722 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.045872 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.046591 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.047523 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.047826 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.047897 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.048032 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.048063 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.048115 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.048201 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.048219 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:32.053550 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.050851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:33969 (local address 127.17.95.194:40297)
0927 11:08:32.050887 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.050889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.051010 (+ 121us) server_negotiation.cc:407] Connection header received
0927 11:08:32.051037 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.051037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.051045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.051131 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.051261 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.051964 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.052721 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.053005 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.053074 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.053277 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.053306 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.053356 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.053427 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.053443 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:08:32.053565 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.050799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:40297 (local address 127.17.95.194:33969)
0927 11:08:32.050948 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.050957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:32.051003 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:32.051143 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:32.051144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:32.051145 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:32.051197 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.051198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.052036 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.052037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.052645 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.052646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.053102 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.053103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.053160 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.053182 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.053370 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.053371 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 11:08:32.053409 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 11:08:32.053443 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260927 11:08:32.055961 25735 raft_consensus.cc:1261] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Refusing update from remote peer 365035a25b204076a930a763c8c23e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.056015 25455 raft_consensus.cc:1261] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Refusing update from remote peer 365035a25b204076a930a763c8c23e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.056174 25836 consensus_queue.cc:1035] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.056263 25810 consensus_queue.cc:1035] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.070561 25298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:32.070655 25298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:32.071241 25298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Sending DeleteTablet for 3 replicas of tablet 49844795c0e5488c8ae82f938d201e7b
I20260927 11:08:32.071465 25575 tablet_service.cc:1514] Processing DeleteTablet for tablet 49844795c0e5488c8ae82f938d201e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49850
I20260927 11:08:32.071477 25435 tablet_service.cc:1514] Processing DeleteTablet for tablet 49844795c0e5488c8ae82f938d201e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260927 11:08:32.071547 25831 tablet_replica.cc:330] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: stopping tablet replica
I20260927 11:08:32.071568 25829 tablet_replica.cc:330] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: stopping tablet replica
I20260927 11:08:32.071587 25831 raft_consensus.cc:2229] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:32.071641 25829 raft_consensus.cc:2229] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:32.071668 25829 raft_consensus.cc:2258] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.071669 25831 raft_consensus.cc:2258] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.071730 25715 tablet_service.cc:1514] Processing DeleteTablet for tablet 49844795c0e5488c8ae82f938d201e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33530
I20260927 11:08:32.071801 25830 tablet_replica.cc:330] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: stopping tablet replica
I20260927 11:08:32.071875 25830 raft_consensus.cc:2229] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:32.071940 25830 raft_consensus.cc:2258] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.072118 25829 ts_tablet_manager.cc:1905] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.072237 25830 ts_tablet_manager.cc:1905] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.073267 25831 ts_tablet_manager.cc:1905] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.073383 25829 ts_tablet_manager.cc:1918] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:32.073426 25829 log.cc:1196] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/49844795c0e5488c8ae82f938d201e7b
I20260927 11:08:32.073669 25829 ts_tablet_manager.cc:1939] T 49844795c0e5488c8ae82f938d201e7b P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting consensus metadata
I20260927 11:08:32.073695 25830 ts_tablet_manager.cc:1918] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:32.073760 25830 log.cc:1196] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/49844795c0e5488c8ae82f938d201e7b
I20260927 11:08:32.073971 25830 ts_tablet_manager.cc:1939] T 49844795c0e5488c8ae82f938d201e7b P 1b15a03c6ef34df290c6c09745cdf34f: Deleting consensus metadata
I20260927 11:08:32.073962 25284 catalog_manager.cc:4841] TS 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775): tablet 49844795c0e5488c8ae82f938d201e7b (table test-table [id=754d2da7d22547b88d1bf69b57a94c13]) successfully deleted
I20260927 11:08:32.074262 25284 catalog_manager.cc:4841] TS 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437): tablet 49844795c0e5488c8ae82f938d201e7b (table test-table [id=754d2da7d22547b88d1bf69b57a94c13]) successfully deleted
I20260927 11:08:32.074743 25831 ts_tablet_manager.cc:1918] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:32.074787 25831 log.cc:1196] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/49844795c0e5488c8ae82f938d201e7b
I20260927 11:08:32.075006 25831 ts_tablet_manager.cc:1939] T 49844795c0e5488c8ae82f938d201e7b P 365035a25b204076a930a763c8c23e4e: Deleting consensus metadata
I20260927 11:08:32.075338 25282 catalog_manager.cc:4841] TS 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297): tablet 49844795c0e5488c8ae82f938d201e7b (table test-table [id=754d2da7d22547b88d1bf69b57a94c13]) successfully deleted
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 11:08:32.097360 25347 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:32.091409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.17.95.254:33789)
0927 11:08:32.091457 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.091459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.091621 (+ 162us) server_negotiation.cc:407] Connection header received
0927 11:08:32.091693 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.091694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.091705 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.091777 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:32.092323 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.092976 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.093850 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.094115 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.094199 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.097286 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36708: BlockingRecv error: recv got EOF from 127.0.0.1:36708 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM
I20260927 11:08:32.105712 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.098311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.17.95.254:33789)
0927 11:08:32.098354 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.098356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.098376 (+ 20us) server_negotiation.cc:407] Connection header received
0927 11:08:32.098430 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.098431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.098440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.098484 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:32.098909 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.099469 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.100107 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.100390 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.100469 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.102968 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:32.102970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:32.102970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:32.102979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:32.104690 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:32.105007 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:32.105008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:32.105008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:32.105078 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:32.105252 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:32.105254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:32.105254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:32.105384 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:32.105484 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.105558 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.105629 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:08:32.107272 25298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:32.107450 25298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:32.108827 25435 tablet_service.cc:1467] Processing CreateTablet for tablet e8bf580b328545b19be2327188961763 (DEFAULT_TABLE table=test-table [id=3da30e270204416aa981390b12d8e3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:32.108850 25715 tablet_service.cc:1467] Processing CreateTablet for tablet e8bf580b328545b19be2327188961763 (DEFAULT_TABLE table=test-table [id=3da30e270204416aa981390b12d8e3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:32.108976 25435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8bf580b328545b19be2327188961763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:32.108989 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8bf580b328545b19be2327188961763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:32.110313 25811 tablet_bootstrap.cc:492] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Bootstrap starting.
I20260927 11:08:32.110342 25575 tablet_service.cc:1467] Processing CreateTablet for tablet e8bf580b328545b19be2327188961763 (DEFAULT_TABLE table=test-table [id=3da30e270204416aa981390b12d8e3b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:32.110459 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8bf580b328545b19be2327188961763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:32.110656 25811 tablet_bootstrap.cc:654] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:32.111176 25807 tablet_bootstrap.cc:492] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Bootstrap starting.
I20260927 11:08:32.111518 25807 tablet_bootstrap.cc:654] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:32.111806 25806 tablet_bootstrap.cc:492] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Bootstrap starting.
I20260927 11:08:32.111975 25811 tablet_bootstrap.cc:492] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: No bootstrap required, opened a new log
I20260927 11:08:32.112018 25811 ts_tablet_manager.cc:1397] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:32.112155 25811 raft_consensus.cc:348] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.112232 25811 raft_consensus.cc:374] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:32.112265 25811 raft_consensus.cc:729] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b15a03c6ef34df290c6c09745cdf34f, State: Initialized, Role: FOLLOWER
I20260927 11:08:32.112305 25811 consensus_queue.cc:260] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.112393 25806 tablet_bootstrap.cc:654] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:32.112409 25811 ts_tablet_manager.cc:1428] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:32.112926 25807 tablet_bootstrap.cc:492] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: No bootstrap required, opened a new log
I20260927 11:08:32.112969 25807 ts_tablet_manager.cc:1397] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:32.113116 25807 raft_consensus.cc:348] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.113185 25807 raft_consensus.cc:374] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:32.113212 25807 raft_consensus.cc:729] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7816405ea2f34b91bf7b32da8e9e8e46, State: Initialized, Role: FOLLOWER
I20260927 11:08:32.113260 25807 consensus_queue.cc:260] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.113277 25806 tablet_bootstrap.cc:492] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: No bootstrap required, opened a new log
I20260927 11:08:32.113327 25806 ts_tablet_manager.cc:1397] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:32.113340 25807 ts_tablet_manager.cc:1428] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:32.113487 25806 raft_consensus.cc:348] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.113562 25806 raft_consensus.cc:374] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:32.113582 25806 raft_consensus.cc:729] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Initialized, Role: FOLLOWER
I20260927 11:08:32.113620 25806 consensus_queue.cc:260] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.113700 25806 ts_tablet_manager.cc:1428] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:32.162137 25859 raft_consensus.cc:482] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:32.162240 25859 raft_consensus.cc:504] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.162411 25859 leader_election.cc:288] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:32.162634 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bf580b328545b19be2327188961763" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f" is_pre_election: true
I20260927 11:08:32.162627 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bf580b328545b19be2327188961763" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" is_pre_election: true
I20260927 11:08:32.162736 25455 raft_consensus.cc:2454] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 0.
I20260927 11:08:32.162737 25735 raft_consensus.cc:2454] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 0.
I20260927 11:08:32.162919 25528 leader_election.cc:302] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365035a25b204076a930a763c8c23e4e, 7816405ea2f34b91bf7b32da8e9e8e46; no voters:
I20260927 11:08:32.163002 25859 raft_consensus.cc:2781] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:32.163038 25859 raft_consensus.cc:482] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:32.163062 25859 raft_consensus.cc:3037] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.163709 25859 raft_consensus.cc:504] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.163832 25859 leader_election.cc:288] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 election: Requested vote from peers 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:32.163996 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bf580b328545b19be2327188961763" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f"
I20260927 11:08:32.164006 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bf580b328545b19be2327188961763" candidate_uuid: "365035a25b204076a930a763c8c23e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7816405ea2f34b91bf7b32da8e9e8e46"
I20260927 11:08:32.164091 25455 raft_consensus.cc:3037] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.164106 25735 raft_consensus.cc:3037] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.164714 25735 raft_consensus.cc:2454] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 1.
I20260927 11:08:32.164857 25455 raft_consensus.cc:2454] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365035a25b204076a930a763c8c23e4e in term 1.
I20260927 11:08:32.164878 25528 leader_election.cc:302] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 365035a25b204076a930a763c8c23e4e; no voters:
I20260927 11:08:32.164978 25859 raft_consensus.cc:2781] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:32.165022 25859 raft_consensus.cc:686] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 1 LEADER]: Becoming Leader. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Running, Role: LEADER
I20260927 11:08:32.165062 25859 consensus_queue.cc:237] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.165622 25296 catalog_manager.cc:5495] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 365035a25b204076a930a763c8c23e4e (127.17.95.194). New cstate: current_term: 1 leader_uuid: "365035a25b204076a930a763c8c23e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:32.170849 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.168464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.17.95.194:40297)
0927 11:08:32.168530 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.168532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.168542 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:32.168564 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.168565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.168575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.168646 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.168758 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.169479 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.170170 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.170459 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.170527 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.170645 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.170676 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.170726 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.170766 (+ 40us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.170783 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:08:32.171505 25735 raft_consensus.cc:1261] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Refusing update from remote peer 365035a25b204076a930a763c8c23e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.171689 25455 raft_consensus.cc:1261] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Refusing update from remote peer 365035a25b204076a930a763c8c23e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.171707 25859 consensus_queue.cc:1035] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.171936 25859 consensus_queue.cc:1035] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.174311 25298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:32.174391 25298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:32.175200 25298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Sending DeleteTablet for 3 replicas of tablet e8bf580b328545b19be2327188961763
I20260927 11:08:32.175371 25575 tablet_service.cc:1514] Processing DeleteTablet for tablet e8bf580b328545b19be2327188961763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49850
I20260927 11:08:32.175563 25715 tablet_service.cc:1514] Processing DeleteTablet for tablet e8bf580b328545b19be2327188961763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33530
I20260927 11:08:32.175558 25574 tablet_service.cc:1514] Processing DeleteTablet for tablet e8bf580b328545b19be2327188961763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49850
I20260927 11:08:32.175638 25831 tablet_replica.cc:330] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: stopping tablet replica
I20260927 11:08:32.175637 25830 tablet_replica.cc:330] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: stopping tablet replica
I20260927 11:08:32.175674 25831 raft_consensus.cc:2229] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:32.175681 25830 raft_consensus.cc:2229] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:32.175709 25831 raft_consensus.cc:2258] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.175727 25830 raft_consensus.cc:2258] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.175904 25435 tablet_service.cc:1514] Processing DeleteTablet for tablet e8bf580b328545b19be2327188961763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260927 11:08:32.175987 25829 tablet_replica.cc:330] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: stopping tablet replica
I20260927 11:08:32.176026 25829 raft_consensus.cc:2229] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:32.176031 25830 ts_tablet_manager.cc:1905] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.176065 25829 raft_consensus.cc:2258] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.176093 25434 tablet_service.cc:1514] Processing DeleteTablet for tablet e8bf580b328545b19be2327188961763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260927 11:08:32.176980 25831 ts_tablet_manager.cc:1905] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.176918 25298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:32.177070 25298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:32.177385 25830 ts_tablet_manager.cc:1918] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:32.177433 25830 log.cc:1196] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/e8bf580b328545b19be2327188961763
I20260927 11:08:32.177639 25830 ts_tablet_manager.cc:1939] T e8bf580b328545b19be2327188961763 P 1b15a03c6ef34df290c6c09745cdf34f: Deleting consensus metadata
I20260927 11:08:32.178069 25284 catalog_manager.cc:4841] TS 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437): tablet e8bf580b328545b19be2327188961763 (table test-table [id=3da30e270204416aa981390b12d8e3b4]) successfully deleted
I20260927 11:08:32.178207 25831 ts_tablet_manager.cc:1918] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:32.178243 25831 log.cc:1196] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/e8bf580b328545b19be2327188961763
I20260927 11:08:32.178376 25831 ts_tablet_manager.cc:1939] T e8bf580b328545b19be2327188961763 P 365035a25b204076a930a763c8c23e4e: Deleting consensus metadata
I20260927 11:08:32.178803 25282 catalog_manager.cc:4841] TS 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297): tablet e8bf580b328545b19be2327188961763 (table test-table [id=3da30e270204416aa981390b12d8e3b4]) successfully deleted
W20260927 11:08:32.178977 25282 catalog_manager.cc:4808] TS 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297): delete failed for tablet e8bf580b328545b19be2327188961763 because the tablet was not found. No further retry: Not found: Tablet not found: e8bf580b328545b19be2327188961763
I20260927 11:08:32.179733 25829 ts_tablet_manager.cc:1905] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.179795 25574 tablet_service.cc:1467] Processing CreateTablet for tablet e9bd3bed396d43119126cbd83956815d (DEFAULT_TABLE table=test-table [id=923e318c68b844c3bf3d92aafb98fa8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:32.179837 25434 tablet_service.cc:1467] Processing CreateTablet for tablet e9bd3bed396d43119126cbd83956815d (DEFAULT_TABLE table=test-table [id=923e318c68b844c3bf3d92aafb98fa8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:32.179888 25574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9bd3bed396d43119126cbd83956815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:32.179939 25434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9bd3bed396d43119126cbd83956815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:32.181185 25829 ts_tablet_manager.cc:1918] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:32.181249 25829 log.cc:1196] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/e8bf580b328545b19be2327188961763
I20260927 11:08:32.181300 25807 tablet_bootstrap.cc:492] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Bootstrap starting.
I20260927 11:08:32.181411 25829 ts_tablet_manager.cc:1939] T e8bf580b328545b19be2327188961763 P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting consensus metadata
I20260927 11:08:32.181663 25807 tablet_bootstrap.cc:654] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:32.182189 25284 catalog_manager.cc:4841] TS 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775): tablet e8bf580b328545b19be2327188961763 (table test-table [id=3da30e270204416aa981390b12d8e3b4]) successfully deleted
W20260927 11:08:32.182252 25284 catalog_manager.cc:4808] TS 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775): delete failed for tablet e8bf580b328545b19be2327188961763 because the tablet was not found. No further retry: Not found: Tablet not found: e8bf580b328545b19be2327188961763
I20260927 11:08:32.182209 25715 tablet_service.cc:1467] Processing CreateTablet for tablet e9bd3bed396d43119126cbd83956815d (DEFAULT_TABLE table=test-table [id=923e318c68b844c3bf3d92aafb98fa8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:32.182327 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9bd3bed396d43119126cbd83956815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:32.182427 25807 tablet_bootstrap.cc:492] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: No bootstrap required, opened a new log
I20260927 11:08:32.182473 25807 ts_tablet_manager.cc:1397] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:32.182610 25807 raft_consensus.cc:348] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.182674 25807 raft_consensus.cc:374] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:32.182694 25807 raft_consensus.cc:729] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7816405ea2f34b91bf7b32da8e9e8e46, State: Initialized, Role: FOLLOWER
I20260927 11:08:32.182736 25807 consensus_queue.cc:260] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.182824 25807 ts_tablet_manager.cc:1428] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:32.183002 25806 tablet_bootstrap.cc:492] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Bootstrap starting.
I20260927 11:08:32.183315 25806 tablet_bootstrap.cc:654] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:32.183562 25811 tablet_bootstrap.cc:492] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Bootstrap starting.
I20260927 11:08:32.183879 25811 tablet_bootstrap.cc:654] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:32.184407 25811 tablet_bootstrap.cc:492] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: No bootstrap required, opened a new log
I20260927 11:08:32.184465 25811 ts_tablet_manager.cc:1397] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:32.184480 25806 tablet_bootstrap.cc:492] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: No bootstrap required, opened a new log
I20260927 11:08:32.184520 25806 ts_tablet_manager.cc:1397] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:32.184579 25811 raft_consensus.cc:348] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.184641 25811 raft_consensus.cc:374] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:32.184620 25806 raft_consensus.cc:348] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.184665 25811 raft_consensus.cc:729] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b15a03c6ef34df290c6c09745cdf34f, State: Initialized, Role: FOLLOWER
I20260927 11:08:32.184670 25806 raft_consensus.cc:374] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:32.184690 25806 raft_consensus.cc:729] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365035a25b204076a930a763c8c23e4e, State: Initialized, Role: FOLLOWER
I20260927 11:08:32.184711 25811 consensus_queue.cc:260] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.184725 25806 consensus_queue.cc:260] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.184796 25806 ts_tablet_manager.cc:1428] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:32.184808 25811 ts_tablet_manager.cc:1428] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:32.215986 25812 raft_consensus.cc:482] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:32.216070 25812 raft_consensus.cc:504] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.216398 25812 leader_election.cc:288] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:32.219411 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.216639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:39275 (local address 127.17.95.195:37437)
0927 11:08:32.216712 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.216714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.216750 (+ 36us) server_negotiation.cc:407] Connection header received
0927 11:08:32.216873 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.216874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.216884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.216930 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:32.217121 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.217823 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.218721 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.219104 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.219168 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.219176 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.219313 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.219350 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 11:08:32.219424 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.216544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:37437 (local address 127.17.95.193:39275)
0927 11:08:32.216680 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.216692 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:32.216792 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:32.216958 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:32.216959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:32.216961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:32.217046 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.217047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.217917 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.217918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.218634 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.218635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.219172 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.219172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.219237 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.219241 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:32.219302 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 11:08:32.219341 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260927 11:08:32.219679 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9bd3bed396d43119126cbd83956815d" candidate_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f" is_pre_election: true
I20260927 11:08:32.219754 25735 raft_consensus.cc:2454] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7816405ea2f34b91bf7b32da8e9e8e46 in term 0.
I20260927 11:08:32.219926 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.216676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:42955 (local address 127.17.95.194:40297)
0927 11:08:32.216742 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.216744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.216751 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:32.216826 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.216827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.216836 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.216895 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:32.217034 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.217788 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.219103 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.219583 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.219649 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.219657 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.219838 (+ 181us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.219872 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 11:08:32.219928 25388 leader_election.cc:302] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 7816405ea2f34b91bf7b32da8e9e8e46; no voters:
I20260927 11:08:32.220109 25812 raft_consensus.cc:2781] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:32.220131 25884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.216544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:40297 (local address 127.17.95.193:42955)
0927 11:08:32.216700 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.216710 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:32.216752 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:32.216899 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:32.216900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:32.216903 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:32.216977 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.216978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.217902 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.217903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.219006 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.219009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.219649 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.219650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.219740 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.219744 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:32.220012 (+ 268us) client_negotiation.cc:240] Negotiation successful
0927 11:08:32.220050 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":64,"thread_start_us":51,"threads_started":1}
I20260927 11:08:32.220155 25812 raft_consensus.cc:482] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:32.220201 25812 raft_consensus.cc:3037] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.220419 25595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9bd3bed396d43119126cbd83956815d" candidate_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "365035a25b204076a930a763c8c23e4e" is_pre_election: true
I20260927 11:08:32.220522 25595 raft_consensus.cc:2454] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7816405ea2f34b91bf7b32da8e9e8e46 in term 0.
I20260927 11:08:32.221065 25812 raft_consensus.cc:504] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.221199 25812 leader_election.cc:288] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [CANDIDATE]: Term 1 election: Requested vote from peers 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297), 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437)
I20260927 11:08:32.221319 25735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9bd3bed396d43119126cbd83956815d" candidate_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b15a03c6ef34df290c6c09745cdf34f"
I20260927 11:08:32.221406 25735 raft_consensus.cc:3037] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.222030 25735 raft_consensus.cc:2454] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7816405ea2f34b91bf7b32da8e9e8e46 in term 1.
I20260927 11:08:32.222153 25595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9bd3bed396d43119126cbd83956815d" candidate_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "365035a25b204076a930a763c8c23e4e"
I20260927 11:08:32.222234 25595 raft_consensus.cc:3037] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:32.222247 25388 leader_election.cc:302] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b15a03c6ef34df290c6c09745cdf34f, 7816405ea2f34b91bf7b32da8e9e8e46; no voters:
I20260927 11:08:32.222338 25812 raft_consensus.cc:2781] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:32.222496 25812 raft_consensus.cc:686] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 LEADER]: Becoming Leader. State: Replica: 7816405ea2f34b91bf7b32da8e9e8e46, State: Running, Role: LEADER
I20260927 11:08:32.222568 25812 consensus_queue.cc:237] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } }
I20260927 11:08:32.222952 25595 raft_consensus.cc:2454] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7816405ea2f34b91bf7b32da8e9e8e46 in term 1.
I20260927 11:08:32.223258 25298 catalog_manager.cc:5495] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193). New cstate: current_term: 1 leader_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7816405ea2f34b91bf7b32da8e9e8e46" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 46775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:32.231777 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.229280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.17.95.193:46775)
0927 11:08:32.229330 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.229332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.229342 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:32.229374 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.229375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.229393 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.229470 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.229550 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.230229 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.230938 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.231218 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.231289 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.231425 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.231565 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.231656 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.231699 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.231716 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 11:08:32.236658 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.234416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:57279 (local address 127.17.95.194:40297)
0927 11:08:32.234454 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.234456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.234482 (+ 26us) server_negotiation.cc:407] Connection header received
0927 11:08:32.234549 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.234549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.234559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.234626 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.234750 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.235209 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.235982 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.236165 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.236245 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.236411 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.236446 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.236519 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.236580 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.236597 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:08:32.236697 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.234257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:40297 (local address 127.17.95.193:57279)
0927 11:08:32.234424 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.234435 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:32.234496 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:32.234628 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:32.234629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:32.234630 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:32.234686 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.234687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.235302 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.235303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.235879 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.235881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.236249 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.236250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.236312 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.236323 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.236523 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.236525 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:32.236576 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 11:08:32.236614 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260927 11:08:32.238627 25595 raft_consensus.cc:1261] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Refusing update from remote peer 7816405ea2f34b91bf7b32da8e9e8e46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.238639 25735 raft_consensus.cc:1261] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Refusing update from remote peer 7816405ea2f34b91bf7b32da8e9e8e46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:32.238879 25812 consensus_queue.cc:1035] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b15a03c6ef34df290c6c09745cdf34f" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.239094 25812 consensus_queue.cc:1035] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "365035a25b204076a930a763c8c23e4e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 40297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:32.246187 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.243591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:37539 (local address 127.17.95.193:46775)
0927 11:08:32.243622 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:32.243624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:32.243630 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:08:32.243649 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:32.243649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:32.243657 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:32.243702 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:32.243830 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.244486 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.245457 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.245732 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.245795 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.245963 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.245995 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.246052 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:32.246110 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 11:08:32.246130 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 11:08:32.246212 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.243515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:46775 (local address 127.17.95.194:37539)
0927 11:08:32.243549 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.243559 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:32.243595 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:32.243737 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:32.243737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:32.243738 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 11:08:32.243771 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.243772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.244571 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.244572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.245370 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:32.245372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:32.245828 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:32.245829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:32.245881 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:32.245887 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 11:08:32.246063 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 11:08:32.246065 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:32.246109 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 11:08:32.246148 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 11:08:32.253113 25298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:32.253206 25298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:32.253809 25298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f2cf97ee694430d8575be3bc896b3e7: Sending DeleteTablet for 3 replicas of tablet e9bd3bed396d43119126cbd83956815d
I20260927 11:08:32.254056 25434 tablet_service.cc:1514] Processing DeleteTablet for tablet e9bd3bed396d43119126cbd83956815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260927 11:08:32.254065 25715 tablet_service.cc:1514] Processing DeleteTablet for tablet e9bd3bed396d43119126cbd83956815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:33530
I20260927 11:08:32.254133 25574 tablet_service.cc:1514] Processing DeleteTablet for tablet e9bd3bed396d43119126cbd83956815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:32 UTC) from {username='oryx', principal='oryx/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49850
I20260927 11:08:32.254184 25829 tablet_replica.cc:330] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: stopping tablet replica
I20260927 11:08:32.254204 25831 tablet_replica.cc:330] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: stopping tablet replica
I20260927 11:08:32.254231 25829 raft_consensus.cc:2229] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:32.254240 25831 raft_consensus.cc:2229] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:32.254269 25831 raft_consensus.cc:2258] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.254305 25829 raft_consensus.cc:2258] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.254374 25830 tablet_replica.cc:330] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: stopping tablet replica
I20260927 11:08:32.254426 25830 raft_consensus.cc:2229] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:32.254467 25830 raft_consensus.cc:2258] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:32.254596 25829 ts_tablet_manager.cc:1905] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.254597 25831 ts_tablet_manager.cc:1905] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.256125 25829 ts_tablet_manager.cc:1918] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:32.256191 25829 log.cc:1196] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/e9bd3bed396d43119126cbd83956815d
I20260927 11:08:32.256335 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 25370
I20260927 11:08:32.256402 25829 ts_tablet_manager.cc:1939] T e9bd3bed396d43119126cbd83956815d P 7816405ea2f34b91bf7b32da8e9e8e46: Deleting consensus metadata
I20260927 11:08:32.257602 25831 ts_tablet_manager.cc:1918] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:32.257663 25831 log.cc:1196] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/e9bd3bed396d43119126cbd83956815d
I20260927 11:08:32.257858 25831 ts_tablet_manager.cc:1939] T e9bd3bed396d43119126cbd83956815d P 365035a25b204076a930a763c8c23e4e: Deleting consensus metadata
I20260927 11:08:32.258136 25282 catalog_manager.cc:4841] TS 365035a25b204076a930a763c8c23e4e (127.17.95.194:40297): tablet e9bd3bed396d43119126cbd83956815d (table test-table [id=923e318c68b844c3bf3d92aafb98fa8a]) successfully deleted
I20260927 11:08:32.258394 25830 ts_tablet_manager.cc:1905] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:32.259866 25830 ts_tablet_manager.cc:1918] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 11:08:32.259941 25830 log.cc:1196] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/e9bd3bed396d43119126cbd83956815d
I20260927 11:08:32.260133 25830 ts_tablet_manager.cc:1939] T e9bd3bed396d43119126cbd83956815d P 1b15a03c6ef34df290c6c09745cdf34f: Deleting consensus metadata
I20260927 11:08:32.260426 25284 catalog_manager.cc:4841] TS 1b15a03c6ef34df290c6c09745cdf34f (127.17.95.195:37437): tablet e9bd3bed396d43119126cbd83956815d (table test-table [id=923e318c68b844c3bf3d92aafb98fa8a]) successfully deleted
W20260927 11:08:32.264166 25284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:46775 (error 108)
I20260927 11:08:32.264619 25894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:32.264471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:46775 (local address 127.0.0.1:34798)
0927 11:08:32.264555 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:32.264589 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:46775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":45,"threads_started":1}
W20260927 11:08:32.264720 25284 catalog_manager.cc:4584] TS 7816405ea2f34b91bf7b32da8e9e8e46 (127.17.95.193:46775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9bd3bed396d43119126cbd83956815d: Network error: Client connection negotiation failed: client connection to 127.17.95.193:46775: connect: Connection refused (error 111)
I20260927 11:08:32.264942 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 25510
I20260927 11:08:32.270872 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 25650
I20260927 11:08:32.276140 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 25266
2026-09-27T11:08:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2610 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25897](info): set up 2 sockets
Sep 27 11:08:32 dist-test-slave-6j0v krb5kdc[25897](info): commencing operation
krb5kdc: starting...
W20260927 11:08:33.283562 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:33 dist-test-slave-6j0v krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:33.396343 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:38257
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:33.479985 25910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:33.480222 25910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:33.480247 25910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:33.481348 25910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:33.481389 25910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:33.481405 25910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:33.481420 25910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:33.481434 25910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:33.482788 25910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:33.482975 25910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:33.483170 25910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:33.487767 25915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:33.487833 25918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:33.487797 25910 server_base.cc:1034] running on GCE node
W20260927 11:08:33.487767 25916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:33.488199 25910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:33.488461 25910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:33.489619 25910 hybrid_clock.cc:648] HybridClock initialized: now 1790507313489604 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T11:08:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1216 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:33 dist-test-slave-6j0v krb5kdc[25923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:33 dist-test-slave-6j0v krb5kdc[25923](info): set up 2 sockets
Sep 27 11:08:33 dist-test-slave-6j0v krb5kdc[25923](info): commencing operation
krb5kdc: starting...
W20260927 11:08:34.519692 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:34 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:34.629359 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:46305
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:46305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:34.710114 25936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:34.710285 25936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:34.710304 25936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:34.711414 25936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:34.711452 25936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:34.711467 25936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:34.711498 25936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:34.711524 25936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:34.713006 25936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:46305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:46305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:34.713223 25936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:34.713433 25936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:34.717902 25944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:34.717911 25942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:34.717971 25936 server_base.cc:1034] running on GCE node
W20260927 11:08:34.717941 25941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:34.718292 25936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:34.718592 25936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:34.719734 25936 hybrid_clock.cc:648] HybridClock initialized: now 1790507314719728 us; error 31 us; skew 500 ppm
Sep 27 11:08:34 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507314, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:34.722461 25936 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:34.722966 25936 webserver.cc:460] Webserver started at http://127.17.95.254:41327/ using document root <none> and password file <none>
I20260927 11:08:34.723218 25936 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:34.723264 25936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:34.723398 25936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:34.724364 25936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "67d355fd09e94797b97ead4e65b1d859"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "28f9a9294acba6699eead6a34b4ed5ab"
server_key_iv: "0ee963c1f2b2ac29b9006b88ca2e9a4f"
server_key_version: "encryptionkey@0"
I20260927 11:08:34.724714 25936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "67d355fd09e94797b97ead4e65b1d859"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "28f9a9294acba6699eead6a34b4ed5ab"
server_key_iv: "0ee963c1f2b2ac29b9006b88ca2e9a4f"
server_key_version: "encryptionkey@0"
I20260927 11:08:34.725998 25936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:34.726657 25951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:34.726809 25936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:34.726882 25936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "67d355fd09e94797b97ead4e65b1d859"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "28f9a9294acba6699eead6a34b4ed5ab"
server_key_iv: "0ee963c1f2b2ac29b9006b88ca2e9a4f"
server_key_version: "encryptionkey@0"
I20260927 11:08:34.726941 25936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:34.748401 25936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:34.750419 25936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:34.750519 25936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:34.754140 25936 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:46305
I20260927 11:08:34.754166 26003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:46305 every 8 connection(s)
I20260927 11:08:34.754464 25936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:34.755273 26004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:34.757316 26004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Bootstrap starting.
I20260927 11:08:34.757862 26004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:34.758152 26004 log.cc:824] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:34.758800 26004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: No bootstrap required, opened a new log
I20260927 11:08:34.760255 26004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:34.760398 26004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:34.760439 26004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d355fd09e94797b97ead4e65b1d859, State: Initialized, Role: FOLLOWER
I20260927 11:08:34.760587 26004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:34.760663 26004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:34.760708 26004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:34.760756 26004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:34.761206 26004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:34.761310 26004 leader_election.cc:302] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67d355fd09e94797b97ead4e65b1d859; no voters:
I20260927 11:08:34.761487 26004 leader_election.cc:288] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:34.761523 26007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:34.761670 26007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 LEADER]: Becoming Leader. State: Replica: 67d355fd09e94797b97ead4e65b1d859, State: Running, Role: LEADER
I20260927 11:08:34.761718 26004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:34.761812 26007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:34.762329 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67d355fd09e94797b97ead4e65b1d859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } } }
I20260927 11:08:34.762331 26009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67d355fd09e94797b97ead4e65b1d859. Latest consensus state: current_term: 1 leader_uuid: "67d355fd09e94797b97ead4e65b1d859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } } }
I20260927 11:08:34.762431 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:34.762435 26009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:34.762619 26016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:34.763139 26016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:34.763602 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 25936
I20260927 11:08:34.763701 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:34.763870 17791 external_mini_cluster.cc:1428] Setting key 02d3830360e18c43b4c0fc896164ff81
I20260927 11:08:34.764492 26016 catalog_manager.cc:1347] Generated new cluster ID: abdc5ec9d2464212bedbbefff6f1a188
I20260927 11:08:34.764554 26016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:08:34 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:34.777660 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:34.764493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.17.95.254:46305)
0927 11:08:34.767313 (+ 2820us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:34.767322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:34.767350 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:08:34.767450 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:34.767457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:34.767506 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:34.767619 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:34.768430 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:34.769741 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:34.770837 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:34.771474 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:34.771566 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:34.774215 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:34.774219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:34.774220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:34.774231 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:34.776094 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:34.776470 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:34.776472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:34.776473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:34.776570 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:34.776781 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:34.776782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:34.776782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:34.777070 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:34.777226 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:34.777300 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 11:08:34.777442 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2781,"thread_start_us":70,"threads_started":1}
I20260927 11:08:34.785555 26016 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:34.786079 26016 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:34.790691 26016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Generated new TSK 0
I20260927 11:08:34.790843 26016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:34.819278 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:46305
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:34.898829 26030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:34.899025 26030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:34.899049 26030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:34.900233 26030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:34.900281 26030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:34.900309 26030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:34.901757 26030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:46305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:34.901985 26030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:34.902222 26030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:34.907127 26035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:34.907174 26030 server_base.cc:1034] running on GCE node
W20260927 11:08:34.907109 26036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:34.907079 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:34.907639 26030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:34.907889 26030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:34.909051 26030 hybrid_clock.cc:648] HybridClock initialized: now 1790507314909039 us; error 31 us; skew 500 ppm
Sep 27 11:08:34 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507314, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:34.911757 26030 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:34.912175 26030 webserver.cc:460] Webserver started at http://127.17.95.193:42917/ using document root <none> and password file <none>
I20260927 11:08:34.912407 26030 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:34.912487 26030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:34.912633 26030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:34.913722 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "b5ad4c2a3c4c4e21a86ce4bf976730a6"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "4f30745caf6aef13b37973b0c417c2ef"
server_key_iv: "d8c5d40790505ce6ebefe04d8d889a64"
server_key_version: "encryptionkey@0"
I20260927 11:08:34.914076 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "b5ad4c2a3c4c4e21a86ce4bf976730a6"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "4f30745caf6aef13b37973b0c417c2ef"
server_key_iv: "d8c5d40790505ce6ebefe04d8d889a64"
server_key_version: "encryptionkey@0"
I20260927 11:08:34.915393 26030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:34.916193 26045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:34.916401 26030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:34.916522 26030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "b5ad4c2a3c4c4e21a86ce4bf976730a6"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "4f30745caf6aef13b37973b0c417c2ef"
server_key_iv: "d8c5d40790505ce6ebefe04d8d889a64"
server_key_version: "encryptionkey@0"
I20260927 11:08:34.916594 26030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:34.947680 26030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:34.949681 26030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:34.949853 26030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:34.950095 26030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:34.950402 26030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:34.950456 26030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:34.950507 26030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:34.950541 26030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:34.956375 26030 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41393
I20260927 11:08:34.956418 26158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41393 every 8 connection(s)
I20260927 11:08:34.956725 26030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:34 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507314, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:34.963869 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26030
I20260927 11:08:34.963999 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:34.964227 17791 external_mini_cluster.cc:1428] Setting key 651a5e768540c5399953599aee3de8c5
I20260927 11:08:34.966706 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:34.957434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:48233)
0927 11:08:34.957582 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:34.957623 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:34.958081 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:34.958294 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:34.958297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:34.958822 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:34.959010 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:34.959014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:34.959680 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:34.959681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:34.960432 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:34.960433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:34.960866 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:34.960867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:34.960978 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:34.961434 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:34.961441 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:34.962957 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:34.965100 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:34.965107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:34.965109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:34.965420 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:34.965671 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:34.965672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:34.965673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:34.965813 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:34.966048 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:34.966054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:34.966213 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 11:08:34.966345 (+ 132us) client_negotiation.cc:240] Negotiation successful
0927 11:08:34.966540 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260927 11:08:34.966727 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:34.957454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:48233 (local address 127.17.95.254:46305)
0927 11:08:34.957527 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:34.957530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:34.957711 (+ 181us) server_negotiation.cc:407] Connection header received
0927 11:08:34.958162 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:34.958162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:34.958174 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:34.958242 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:34.959114 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:34.959604 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:34.960539 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:34.960758 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:34.960826 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:34.963043 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:34.963044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:34.963045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:34.963051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:34.965011 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:34.965524 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:34.965525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:34.965525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:34.965594 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:34.965875 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:34.965876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:34.965876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:34.965985 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:34.966038 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:34.966497 (+ 459us) server_negotiation.cc:299] Negotiation successful
0927 11:08:34.966599 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 11:08:34.967702 26159 heartbeater.cc:347] Connected to a master server at 127.17.95.254:46305
I20260927 11:08:34.967808 26159 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:34.967954 26159 heartbeater.cc:511] Master 127.17.95.254:46305 requested a full tablet report, sending...
I20260927 11:08:34.968448 25968 ts_manager.cc:194] Registered new tserver with Master: b5ad4c2a3c4c4e21a86ce4bf976730a6 (127.17.95.193:41393)
I20260927 11:08:34.969049 25968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:48233
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:34.989401 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:46305
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:35.066231 26164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:35.066413 26164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:35.066434 26164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:35.067605 26164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:35.067644 26164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:35.067669 26164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:35.069056 26164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:46305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:35.069545 26164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:35.069820 26164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:35.074765 26169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.074779 26172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.074783 26170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.075121 26164 server_base.cc:1034] running on GCE node
I20260927 11:08:35.075315 26164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:35.075528 26164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:35.076679 26164 hybrid_clock.cc:648] HybridClock initialized: now 1790507315076665 us; error 27 us; skew 500 ppm
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:35.079023 26164 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:35.079464 26164 webserver.cc:460] Webserver started at http://127.17.95.194:38143/ using document root <none> and password file <none>
I20260927 11:08:35.079691 26164 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:35.079754 26164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:35.079900 26164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:35.080860 26164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "dbbdf67315a144808aab93bc2273b9a1"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "675d6d5c8eeda0f9bc49e1ddf5bbd9d2"
server_key_iv: "4ba26989d5f485578b1d37a4df84a6cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.081194 26164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "dbbdf67315a144808aab93bc2273b9a1"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "675d6d5c8eeda0f9bc49e1ddf5bbd9d2"
server_key_iv: "4ba26989d5f485578b1d37a4df84a6cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.082415 26164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:35.083289 26179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.083487 26164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:35.083592 26164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "dbbdf67315a144808aab93bc2273b9a1"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "675d6d5c8eeda0f9bc49e1ddf5bbd9d2"
server_key_iv: "4ba26989d5f485578b1d37a4df84a6cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.083683 26164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:35.093909 26164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:35.095621 26164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:35.095770 26164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:35.095964 26164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:35.096240 26164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:35.096294 26164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.096325 26164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:35.096351 26164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.102067 26164 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:35697
I20260927 11:08:35.102111 26292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:35697 every 8 connection(s)
I20260927 11:08:35.102391 26164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:35.103348 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26164
I20260927 11:08:35.103483 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:35.103667 17791 external_mini_cluster.cc:1428] Setting key 4d774776a4c78ad39663cbf7df91f3f8
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:35.112182 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.103028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:60593)
0927 11:08:35.103273 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.103324 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.103937 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.104196 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.104200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.104955 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.105256 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.105262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.105928 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.105930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.106729 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.106731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.107267 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.107268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.107376 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.107831 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.107838 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.109174 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:35.110941 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:35.110946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:35.110947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:35.111193 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:35.111382 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:35.111383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:35.111384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:35.111465 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:35.111759 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:35.111763 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:35.111852 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 11:08:35.111935 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 11:08:35.112045 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":151,"threads_started":1}
I20260927 11:08:35.112188 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.103096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:60593 (local address 127.17.95.254:46305)
0927 11:08:35.103172 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.103175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.103447 (+ 272us) server_negotiation.cc:407] Connection header received
0927 11:08:35.104040 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.104041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.104052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.104113 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.105353 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.105829 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.106821 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.107166 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.107262 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.109267 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:35.109268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:35.109269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:35.109277 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:35.110869 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:35.111267 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:35.111268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:35.111268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:35.111319 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:35.111543 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:35.111544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:35.111545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:35.111694 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:35.111758 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:35.111991 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 11:08:35.112092 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
I20260927 11:08:35.112751 26293 heartbeater.cc:347] Connected to a master server at 127.17.95.254:46305
I20260927 11:08:35.112849 26293 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:35.113004 26293 heartbeater.cc:511] Master 127.17.95.254:46305 requested a full tablet report, sending...
I20260927 11:08:35.113305 25968 ts_manager.cc:194] Registered new tserver with Master: dbbdf67315a144808aab93bc2273b9a1 (127.17.95.194:35697)
I20260927 11:08:35.113703 25968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:60593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:35.127859 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:46305
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:35.205715 26298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:35.205960 26298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:35.206009 26298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:35.207315 26298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:35.207376 26298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:35.207417 26298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:35.208899 26298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:46305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:35.209190 26298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:35.209417 26298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:35.214303 26304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.214351 26306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.214303 26298 server_base.cc:1034] running on GCE node
W20260927 11:08:35.214304 26303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.214668 26298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:35.214931 26298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:35.216068 26298 hybrid_clock.cc:648] HybridClock initialized: now 1790507315216053 us; error 30 us; skew 500 ppm
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:35.218423 26298 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:35.218834 26298 webserver.cc:460] Webserver started at http://127.17.95.195:40073/ using document root <none> and password file <none>
I20260927 11:08:35.219058 26298 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:35.219164 26298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:35.219302 26298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:35.220469 26298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "9f31c4ae89a845d0847d1b6fef8174c0"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "11910840b66c20fa17320e68c09c0c72"
server_key_iv: "c0e2802d4b8ee4b77c3f6c062b335597"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.220813 26298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "9f31c4ae89a845d0847d1b6fef8174c0"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "11910840b66c20fa17320e68c09c0c72"
server_key_iv: "c0e2802d4b8ee4b77c3f6c062b335597"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.222152 26298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:35.223217 26313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.223518 26298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:35.223645 26298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "9f31c4ae89a845d0847d1b6fef8174c0"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "11910840b66c20fa17320e68c09c0c72"
server_key_iv: "c0e2802d4b8ee4b77c3f6c062b335597"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.223737 26298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:35.238757 26298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:35.241369 26298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:35.241520 26298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:35.241757 26298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:35.242159 26298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:35.242214 26298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.242246 26298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:35.242278 26298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.248576 26298 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:39787
I20260927 11:08:35.248636 26426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:39787 every 8 connection(s)
I20260927 11:08:35.249033 26298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:35.252099 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26298
I20260927 11:08:35.252198 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:35.252411 17791 external_mini_cluster.cc:1428] Setting key 3bbb226a9c460ad03d182442eab62658
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:35.260176 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.249856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:46265 (local address 127.17.95.254:46305)
0927 11:08:35.249946 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.249948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.250140 (+ 192us) server_negotiation.cc:407] Connection header received
0927 11:08:35.250811 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.250812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.250828 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.250922 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.252331 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.253095 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.254491 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.254781 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.254836 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.257082 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:35.257084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:35.257084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:35.257093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:35.258813 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:35.259282 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:35.259283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:35.259284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:35.259341 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:35.259575 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:35.259576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:35.259576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:35.259689 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:35.259760 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:35.260017 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 11:08:35.260089 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 11:08:35.260373 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.249844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:46265)
0927 11:08:35.250014 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.250055 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.250710 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.251008 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.251012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.251934 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.252242 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.252249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.253179 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.253180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.254381 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.254396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.254930 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.254931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.255106 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.255636 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.255645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.256982 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:35.258897 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:35.258903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:35.258905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:35.259201 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:35.259402 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:35.259403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:35.259403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:35.259507 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:35.259752 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:35.259758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:35.259886 (+ 128us) client_negotiation.cc:769] Sending connection context
0927 11:08:35.259980 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 11:08:35.260138 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260927 11:08:35.260798 26427 heartbeater.cc:347] Connected to a master server at 127.17.95.254:46305
I20260927 11:08:35.260895 26427 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:35.261060 26427 heartbeater.cc:511] Master 127.17.95.254:46305 requested a full tablet report, sending...
I20260927 11:08:35.261431 25968 ts_manager.cc:194] Registered new tserver with Master: 9f31c4ae89a845d0847d1b6fef8174c0 (127.17.95.195:39787)
I20260927 11:08:35.261844 25968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:46265
I20260927 11:08:35.264050 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab.
Entry for principal oryx/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab.
WARNING: no policy specified for oryx/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab.
Entry for principal oryx/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab.
WARNING: no policy specified for oryx/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab.
Entry for principal oryx/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab.
I20260927 11:08:35.329730 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26030
I20260927 11:08:35.334211 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26164
I20260927 11:08:35.338348 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26298
I20260927 11:08:35.342324 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 25936
I20260927 11:08:35.347487 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:46305
--webserver_interface=127.17.95.254
--webserver_port=41327
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:46305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:35.429687 26436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:35.429908 26436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:35.429946 26436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:35.431109 26436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:35.431177 26436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:35.431205 26436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:35.431241 26436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:35.431272 26436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:35.432674 26436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:46305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:46305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=41327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:35.432932 26436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:35.433163 26436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:35.437834 26444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.437888 26442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.438007 26436 server_base.cc:1034] running on GCE node
W20260927 11:08:35.437836 26441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.438308 26436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:35.438609 26436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:35.439783 26436 hybrid_clock.cc:648] HybridClock initialized: now 1790507315439773 us; error 31 us; skew 500 ppm
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:35.442337 26436 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:35.442725 26436 webserver.cc:460] Webserver started at http://127.17.95.254:41327/ using document root <none> and password file <none>
I20260927 11:08:35.442955 26436 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:35.443034 26436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:35.444422 26436 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:35.445147 26451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.445331 26436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:35.445425 26436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "67d355fd09e94797b97ead4e65b1d859"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "28f9a9294acba6699eead6a34b4ed5ab"
server_key_iv: "0ee963c1f2b2ac29b9006b88ca2e9a4f"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.445765 26436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:35.455124 26436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:35.456995 26436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:35.457158 26436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:35.461082 26436 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:46305
I20260927 11:08:35.461158 26503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:46305 every 8 connection(s)
I20260927 11:08:35.461524 26436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:35.461714 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26436
I20260927 11:08:35.462035 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:41393
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:46305
--webserver_port=42917
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:08:35.462977 26504 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:08:35.463620 26504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Bootstrap starting.
I20260927 11:08:35.471686 26504 log.cc:824] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:35.473268 26504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:08:35.473539 26504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Bootstrap complete.
I20260927 11:08:35.475797 26504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:35.476037 26504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d355fd09e94797b97ead4e65b1d859, State: Initialized, Role: FOLLOWER
I20260927 11:08:35.476248 26504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:35.476342 26504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:35.476394 26504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:35.476442 26504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 1 FOLLOWER]: Advancing to term 2
I20260927 11:08:35.477424 26504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:35.477540 26504 leader_election.cc:302] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67d355fd09e94797b97ead4e65b1d859; no voters:
I20260927 11:08:35.477725 26504 leader_election.cc:288] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 11:08:35.477756 26508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 11:08:35.477896 26508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [term 2 LEADER]: Becoming Leader. State: Replica: 67d355fd09e94797b97ead4e65b1d859, State: Running, Role: LEADER
I20260927 11:08:35.478008 26504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:35.477986 26508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } }
I20260927 11:08:35.478351 26510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67d355fd09e94797b97ead4e65b1d859. Latest consensus state: current_term: 2 leader_uuid: "67d355fd09e94797b97ead4e65b1d859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } } }
I20260927 11:08:35.478523 26510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:35.478739 26509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "67d355fd09e94797b97ead4e65b1d859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d355fd09e94797b97ead4e65b1d859" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 46305 } } }
I20260927 11:08:35.478830 26509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:35.479846 26524 catalog_manager.cc:1259] Loaded cluster ID: abdc5ec9d2464212bedbbefff6f1a188
I20260927 11:08:35.479905 26524 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: loading cluster ID for follower catalog manager: success
I20260927 11:08:35.480684 26524 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: acquiring CA information for follower catalog manager: success
I20260927 11:08:35.480975 26524 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:08:35.481060 26515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:35.481205 26515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:35.481299 26515 catalog_manager.cc:1259] Loaded cluster ID: abdc5ec9d2464212bedbbefff6f1a188
I20260927 11:08:35.481333 26515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:35.481496 26515 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:35.481600 26515 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 67d355fd09e94797b97ead4e65b1d859: Loaded TSK: 0
I20260927 11:08:35.481732 26515 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 11:08:35.553906 26506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:35.554136 26506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:35.554169 26506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:35.555434 26506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:35.555500 26506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:35.555543 26506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:35.557031 26506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.193.keytab
--principal=oryx/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:41393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=42917
--tserver_master_addrs=127.17.95.254:46305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:35.557289 26506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:35.557498 26506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:35.562207 26530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.562325 26506 server_base.cc:1034] running on GCE node
W20260927 11:08:35.562211 26533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.562434 26531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.562654 26506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:35.562875 26506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:35.564044 26506 hybrid_clock.cc:648] HybridClock initialized: now 1790507315564033 us; error 34 us; skew 500 ppm
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:35.566581 26506 init.cc:377] Logged in from keytab as oryx/127.17.95.193@KRBTEST.COM (short username oryx)
I20260927 11:08:35.567054 26506 webserver.cc:460] Webserver started at http://127.17.95.193:42917/ using document root <none> and password file <none>
I20260927 11:08:35.567327 26506 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:35.567407 26506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:35.568727 26506 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:35.569360 26540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.569540 26506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:35.569633 26506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "b5ad4c2a3c4c4e21a86ce4bf976730a6"
format_stamp: "Formatted at 2026-09-27 11:08:34 on dist-test-slave-6j0v"
server_key: "4f30745caf6aef13b37973b0c417c2ef"
server_key_iv: "d8c5d40790505ce6ebefe04d8d889a64"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.569947 26506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:35.582916 26506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:35.585167 26506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:35.585318 26506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:35.585541 26506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:35.585817 26506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:35.585868 26506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.585901 26506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:35.585927 26506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.593091 26506 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41393
I20260927 11:08:35.593168 26653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41393 every 8 connection(s)
I20260927 11:08:35.593482 26506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:35.596836 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26506
I20260927 11:08:35.597129 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:35697
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:46305
--webserver_port=38143
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.602267 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.594129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:50553)
0927 11:08:35.594273 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.594308 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.594776 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.595285 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.595288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.595764 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.595922 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.595926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.597563 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.597564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.598749 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.598752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.599415 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.599416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.599534 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.599985 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.599992 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.602070 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
W20260927 11:08:35.602355 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.594211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:50553 (local address 127.17.95.254:46305)
0927 11:08:35.594427 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.594435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.594468 (+ 33us) server_negotiation.cc:407] Connection header received
0927 11:08:35.595002 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.595009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.595075 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.595232 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.596077 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.597442 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.598837 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.599321 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.599436 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.602176 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:50553: BlockingRecv error: recv got EOF from 127.17.95.193:50553 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
W20260927 11:08:35.602749 26654 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:46305 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.95.254:46305: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.607053 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.602997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:54363)
0927 11:08:35.603090 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.603099 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.603139 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.603273 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.603273 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.603396 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.603432 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.603434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.604033 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.604034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.604560 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.604561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.604898 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.604898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.604960 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.605339 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.605344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.606980 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260927 11:08:35.607131 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.603041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:54363 (local address 127.17.95.254:46305)
0927 11:08:35.603093 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.603095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.603127 (+ 32us) server_negotiation.cc:407] Connection header received
0927 11:08:35.603189 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.603190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.603199 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.603264 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.603519 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.603951 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.604640 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.604822 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.604889 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.607049 (+ 2160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:54363: BlockingRecv error: recv got EOF from 127.17.95.193:54363 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.611370 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.607623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:36019)
0927 11:08:35.607658 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.607668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.607703 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.607827 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.607828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.607949 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.607977 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.607978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.608566 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.608567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.609057 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.609058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.609380 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.609381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.609429 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.609794 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.609798 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.611302 (+ 1504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260927 11:08:35.611414 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.607666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:36019 (local address 127.17.95.254:46305)
0927 11:08:35.607699 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.607700 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.607708 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:35.607761 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.607761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.607770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.607823 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.608055 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.608490 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.609135 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.609313 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.609382 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.611360 (+ 1978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:36019: BlockingRecv error: recv got EOF from 127.17.95.193:36019 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 11:08:35.611693 26654 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:08:35.685453 26658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:35.685653 26658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:35.685675 26658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:35.686761 26658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:35.686793 26658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:35.686818 26658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:35.688220 26658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.194.keytab
--principal=oryx/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:35697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=38143
--tserver_master_addrs=127.17.95.254:46305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:35.688419 26658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:35.688591 26658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:35.693362 26666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.693372 26664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.693387 26658 server_base.cc:1034] running on GCE node
W20260927 11:08:35.693396 26663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.693732 26658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:35.693984 26658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:35.695133 26658 hybrid_clock.cc:648] HybridClock initialized: now 1790507315695133 us; error 39 us; skew 500 ppm
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:35.697456 26658 init.cc:377] Logged in from keytab as oryx/127.17.95.194@KRBTEST.COM (short username oryx)
I20260927 11:08:35.697850 26658 webserver.cc:460] Webserver started at http://127.17.95.194:38143/ using document root <none> and password file <none>
I20260927 11:08:35.698067 26658 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:35.698145 26658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:35.699512 26658 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:35.700016 26673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.700165 26658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:35.700259 26658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "dbbdf67315a144808aab93bc2273b9a1"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "675d6d5c8eeda0f9bc49e1ddf5bbd9d2"
server_key_iv: "4ba26989d5f485578b1d37a4df84a6cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.700559 26658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:35.721182 26658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:35.722738 26658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:35.722913 26658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:35.723145 26658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:35.723435 26658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:35.723495 26658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.723529 26658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:35.723555 26658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.729234 26658 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:35697
I20260927 11:08:35.729286 26786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:35697 every 8 connection(s)
I20260927 11:08:35.729576 26658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:35.732452 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26658
I20260927 11:08:35.732731 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:39787
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:46305
--webserver_port=40073
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 11:08:35.737051 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.730227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:43353 (local address 127.17.95.254:46305)
0927 11:08:35.730268 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.730270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.730476 (+ 206us) server_negotiation.cc:407] Connection header received
0927 11:08:35.730953 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.730954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.730968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.731019 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.731912 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.732478 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.733661 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.733903 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.733980 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.736999 (+ 3019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:43353: BlockingRecv error: recv got EOF from 127.17.95.194:43353 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:35.737166 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.730206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:43353)
0927 11:08:35.730365 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.730395 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.730860 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.731131 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.731134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.731646 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.731809 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.731813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.732582 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.732584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.733575 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.733578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.733986 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.733987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.734120 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.734562 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.734569 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.736933 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
W20260927 11:08:35.737360 26787 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:46305 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.95.254:46305: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.741834 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.737553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:42809)
0927 11:08:35.737602 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.737613 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.737652 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.737869 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.737871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.738032 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.738084 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.738086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.738661 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.738662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.739518 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.739521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.739802 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.739803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.739874 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.740184 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.740189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.741769 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 11:08:35.741878 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.737611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:42809 (local address 127.17.95.254:46305)
0927 11:08:35.737767 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.737768 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.737775 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:35.737793 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.737793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.737800 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.737846 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.738158 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.738592 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.739590 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.739737 (+ 147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.739781 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.741824 (+ 2043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:42809: BlockingRecv error: recv got EOF from 127.17.95.194:42809 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.745850 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.742085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:34567)
0927 11:08:35.742126 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.742136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.742171 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.742304 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.742304 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.742393 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.742414 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.742415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.743236 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.743237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.743622 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.743623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.743997 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.743998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.744026 (+ 28us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.744371 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.744374 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.745793 (+ 1419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 11:08:35.745891 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.742147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:34567 (local address 127.17.95.254:46305)
0927 11:08:35.742197 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.742199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.742207 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:35.742228 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.742228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.742237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.742297 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.742474 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.743172 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.743686 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.743939 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.743995 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.745860 (+ 1865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:34567: BlockingRecv error: recv got EOF from 127.17.95.194:34567 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260927 11:08:35.746104 26787 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:08:35.811342 26790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:35.811511 26790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:35.811532 26790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:35.812580 26790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:35.812614 26790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:35.812638 26790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:35.813952 26790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/krb5kdc/oryx_127.17.95.195.keytab
--principal=oryx/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:39787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=40073
--tserver_master_addrs=127.17.95.254:46305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:35.814116 26790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:35.814256 26790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:35.818733 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:35.818782 26798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.818917 26790 server_base.cc:1034] running on GCE node
W20260927 11:08:35.818791 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:35.819221 26790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:35.819461 26790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:35.820609 26790 hybrid_clock.cc:648] HybridClock initialized: now 1790507315820592 us; error 36 us; skew 500 ppm
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507315, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:35.822803 26790 init.cc:377] Logged in from keytab as oryx/127.17.95.195@KRBTEST.COM (short username oryx)
I20260927 11:08:35.823204 26790 webserver.cc:460] Webserver started at http://127.17.95.195:40073/ using document root <none> and password file <none>
I20260927 11:08:35.823386 26790 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:35.823433 26790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:35.824638 26790 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:35.825172 26805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.825313 26790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:35.825387 26790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "9f31c4ae89a845d0847d1b6fef8174c0"
format_stamp: "Formatted at 2026-09-27 11:08:35 on dist-test-slave-6j0v"
server_key: "11910840b66c20fa17320e68c09c0c72"
server_key_iv: "c0e2802d4b8ee4b77c3f6c062b335597"
server_key_version: "encryptionkey@0"
I20260927 11:08:35.825662 26790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:35.832587 26790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:35.834103 26790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:35.834197 26790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:35.834394 26790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:35.834625 26790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:35.834653 26790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.834695 26790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:35.834720 26790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:35.840690 26790 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:39787
I20260927 11:08:35.840765 26918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:39787 every 8 connection(s)
I20260927 11:08:35.841068 26790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.847268 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26790
W20260927 11:08:35.847869 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.841825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:60299 (local address 127.17.95.254:46305)
0927 11:08:35.841890 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.841892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.842085 (+ 193us) server_negotiation.cc:407] Connection header received
0927 11:08:35.842559 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.842560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.842572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.842620 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.843538 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.844048 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.844926 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.845121 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.845187 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.847803 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:60299: BlockingRecv error: recv got EOF from 127.17.95.195:60299 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:08:35.847906 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.841828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:60299)
0927 11:08:35.841979 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.842011 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.842447 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.842680 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.842684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.843253 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.843421 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.843425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.844128 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.844128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.844831 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.844833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.845202 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.845203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.845308 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.845671 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.845678 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.847699 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
W20260927 11:08:35.848131 26919 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:46305 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.95.254:46305: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.852577 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.848325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:50971)
0927 11:08:35.848371 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.848382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.848418 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.848550 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.848551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.848669 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.848705 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.848706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.849426 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.849426 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.849893 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.849894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.850263 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.850264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.850319 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.850731 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.850736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.852513 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 11:08:35.852658 26923 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.848377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:50971 (local address 127.17.95.254:46305)
0927 11:08:35.848416 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.848418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.848427 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:35.848469 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.848470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.848487 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.848547 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.848773 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.849342 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.849970 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.850189 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.850262 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.852586 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:50971: BlockingRecv error: recv got EOF from 127.17.95.195:50971 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:08:35.854840 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.847832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40202 (local address 127.17.95.254:46305)
0927 11:08:35.847983 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.847985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.847992 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:35.848044 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.848045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.848055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.848085 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.848873 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.849330 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.850139 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.850318 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.850391 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.851632 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:35.851641 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:35.851642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:35.851651 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:35.853543 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:35.853867 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:35.853870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:35.853870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:35.853992 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:35.854207 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:35.854208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:35.854208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:35.854402 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:35.854522 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:35.854674 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 11:08:35.854747 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":46,"threads_started":1}
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:35 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:35.857072 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:35.852788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:57055)
0927 11:08:35.852893 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:35.852902 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:35.852948 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:35.853068 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:35.853069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:35.853161 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:35.853185 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.853186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.853991 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.853991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.854742 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:35.854744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.855147 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.855148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:35.855200 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.855562 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:35.855567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:35.857032 (+ 1465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
W20260927 11:08:35.857198 26923 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:35.852821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:57055 (local address 127.17.95.254:46305)
0927 11:08:35.852849 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:35.852850 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:35.852950 (+ 100us) server_negotiation.cc:407] Connection header received
0927 11:08:35.852999 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:35.853000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:35.853009 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:35.853057 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:35.853247 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.853910 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.854814 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:35.855046 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:35.855149 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:35.857138 (+ 1989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:57055: BlockingRecv error: recv got EOF from 127.17.95.195:57055 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260927 11:08:35.857316 26919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 11:08:36 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:36 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:36.622053 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:36.612066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:49073)
0927 11:08:36.612219 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:36.612230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:36.612307 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:36.612450 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:36.612470 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:36.612628 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:36.612678 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:36.612680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.613405 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.613406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:36.613989 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:36.613991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.614360 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.614361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:36.614398 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:36.620197 (+ 5799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:36.620202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:36.621967 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
W20260927 11:08:36.622169 26926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:36.612175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:49073 (local address 127.17.95.254:46305)
0927 11:08:36.612320 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:36.612322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:36.612330 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:36.612352 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:36.612366 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:36.612376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:36.612432 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:36.612770 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.613330 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.614065 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.614291 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.614346 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:36.622082 (+ 7736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:49073: BlockingRecv error: recv got EOF from 127.17.95.193:49073 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
Sep 27 11:08:36 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:36 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:36.751322 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:36.746455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:49267)
0927 11:08:36.746594 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:36.746604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:36.746656 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:36.746776 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:36.746777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:36.746922 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:36.746970 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:36.746971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.747900 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.747901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:36.748452 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:36.748454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.748875 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.748876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:36.748916 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:36.749341 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:36.749345 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:36.751215 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260927 11:08:36.751360 26926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:36.746535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:49267 (local address 127.17.95.254:46305)
0927 11:08:36.746601 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:36.746603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:36.746644 (+ 41us) server_negotiation.cc:407] Connection header received
0927 11:08:36.746707 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:36.746708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:36.746718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:36.746757 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:36.747029 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.747822 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.748536 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.748819 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.748869 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:36.751296 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:49267: BlockingRecv error: recv got EOF from 127.17.95.194:49267 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 27 11:08:36 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:36 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:36.861896 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:36.857696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:53879)
0927 11:08:36.857827 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:36.857836 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:36.857891 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:36.858014 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:36.858015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:36.858165 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:36.858213 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:36.858215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.858860 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.858861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:36.859405 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:36.859407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.859720 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.859720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:36.859756 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:36.860213 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:36.860217 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:36.861836 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
W20260927 11:08:36.861959 26926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:36.857758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:53879 (local address 127.17.95.254:46305)
0927 11:08:36.857825 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:36.857828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:36.857896 (+ 68us) server_negotiation.cc:407] Connection header received
0927 11:08:36.857931 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:36.857932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:36.857945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:36.857998 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:36.858297 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.858775 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.859481 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:36.859662 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:36.859709 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:36.861916 (+ 2207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:53879: BlockingRecv error: recv got EOF from 127.17.95.195:53879 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 11:08:37 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:37 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:37.627180 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:37.622603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:39411)
0927 11:08:37.622758 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:37.622770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:37.622816 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:37.622941 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:37.622942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:37.623166 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:37.623216 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:37.623218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.623853 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.623853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:37.624514 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:37.624516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.624851 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.624851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:37.624888 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:37.625345 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:37.625349 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:37.627115 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
W20260927 11:08:37.627259 26930 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:37.622685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:39411 (local address 127.17.95.254:46305)
0927 11:08:37.622792 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:37.622794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:37.622815 (+ 21us) server_negotiation.cc:407] Connection header received
0927 11:08:37.622864 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:37.622864 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:37.622873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:37.622921 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:37.623283 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.623785 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.624588 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.624784 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.624845 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:37.627194 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:39411: BlockingRecv error: recv got EOF from 127.17.95.193:39411 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":56,"threads_started":1}
Sep 27 11:08:37 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:37 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:37.755895 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:37.751795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:55285)
0927 11:08:37.751944 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:37.751955 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:37.752038 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:37.752181 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:37.752182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:37.752339 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:37.752382 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:37.752383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.753008 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.753009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:37.753490 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:37.753491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.753828 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.753829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:37.753885 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:37.754264 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:37.754267 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:37.755844 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
W20260927 11:08:37.756012 26930 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:37.751874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:55285 (local address 127.17.95.254:46305)
0927 11:08:37.751938 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:37.751940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:37.752021 (+ 81us) server_negotiation.cc:407] Connection header received
0927 11:08:37.752103 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:37.752104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:37.752116 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:37.752168 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:37.752459 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.752928 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.753566 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.753755 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.753828 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:37.755941 (+ 2113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:55285: BlockingRecv error: recv got EOF from 127.17.95.194:55285 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 27 11:08:37 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:37 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:37.866953 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:37.862459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:47313)
0927 11:08:37.862619 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:37.862632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:37.862694 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:37.862837 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:37.862838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:37.863007 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:37.863056 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:37.863058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.863711 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.863712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:37.864224 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:37.864225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.864576 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.864578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:37.864641 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:37.865033 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:37.865038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:37.866899 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
W20260927 11:08:37.867089 26930 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:37.862574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:47313 (local address 127.17.95.254:46305)
0927 11:08:37.862639 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:37.862642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:37.862677 (+ 35us) server_negotiation.cc:407] Connection header received
0927 11:08:37.862745 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:37.862745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:37.862760 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:37.862830 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:37.863152 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.863620 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.864323 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:37.864495 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:37.864571 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:37.866983 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:47313: BlockingRecv error: recv got EOF from 127.17.95.195:47313 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 11:08:38 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:38 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:38.632225 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:38.627635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:45255)
0927 11:08:38.627770 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:38.627782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:38.627844 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:38.628062 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:38.628063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:38.628272 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:38.628349 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:38.628351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.629093 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.629094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:38.629651 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:38.629653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.629962 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.629963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:38.630001 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:38.630397 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:38.630401 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:38.632139 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
W20260927 11:08:38.632267 26934 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:38.627701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45255 (local address 127.17.95.254:46305)
0927 11:08:38.627852 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:38.627854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:38.627865 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:38.627917 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:38.627918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:38.627930 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:38.628041 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:38.628427 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.629011 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.629717 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.629906 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.629954 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:38.632204 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:45255: BlockingRecv error: recv got EOF from 127.17.95.193:45255 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
Sep 27 11:08:38 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:38 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:38.760756 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:38.756396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:33393)
0927 11:08:38.756525 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:38.756536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:38.756597 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:38.756757 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:38.756758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:38.756906 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:38.756953 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:38.756954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.757593 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.757594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:38.758109 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:38.758111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.758418 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.758419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:38.758458 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:38.758881 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:38.758885 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:38.760673 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260927 11:08:38.760818 26934 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:38.756430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:33393 (local address 127.17.95.254:46305)
0927 11:08:38.756473 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:38.756475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:38.756580 (+ 105us) server_negotiation.cc:407] Connection header received
0927 11:08:38.756668 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:38.756669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:38.756681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:38.756743 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:38.757026 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.757511 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.758180 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.758359 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.758406 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:38.760760 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:33393: BlockingRecv error: recv got EOF from 127.17.95.194:33393 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 27 11:08:38 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:38 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:38.871727 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:38.867410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:38177)
0927 11:08:38.867535 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:38.867547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:38.867607 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:38.867714 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:38.867715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:38.867857 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:38.867903 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:38.867905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.868501 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.868502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:38.869015 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:38.869017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.869310 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.869310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:38.869358 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:38.869738 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:38.869741 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:38.871671 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
W20260927 11:08:38.871871 26934 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:38.867513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:38177 (local address 127.17.95.254:46305)
0927 11:08:38.867586 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:38.867587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:38.867596 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:38.867649 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:38.867649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:38.867660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:38.867700 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:38.867955 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.868424 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.869082 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:38.869252 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:38.869298 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:38.871802 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:38177: BlockingRecv error: recv got EOF from 127.17.95.195:38177 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 27 11:08:39 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:39 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:39.637384 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:39.632724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:45711)
0927 11:08:39.632875 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:39.632886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:39.632935 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:39.633068 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:39.633068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:39.633221 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:39.633269 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:39.633271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.633965 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.633966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:39.634482 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:39.634484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.634815 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.634816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:39.634853 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:39.635471 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:39.635476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:39.637328 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
W20260927 11:08:39.637532 26938 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:39.632799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45711 (local address 127.17.95.254:46305)
0927 11:08:39.632942 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:39.632944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:39.632953 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:39.632975 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:39.632975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:39.632987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:39.633046 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:39.633349 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.633901 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.634554 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.634750 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.634802 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:39.637433 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:45711: BlockingRecv error: recv got EOF from 127.17.95.193:45711 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
Sep 27 11:08:39 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:39 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:39.766052 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:39.761226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:40809)
0927 11:08:39.761394 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:39.761405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:39.761478 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:39.761600 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:39.761601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:39.761749 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:39.761797 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:39.761798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.762450 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.762451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:39.763011 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:39.763012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.763513 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.763514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:39.763596 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:39.764235 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:39.764240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:39.765996 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
W20260927 11:08:39.766196 26938 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:39.761353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:40809 (local address 127.17.95.254:46305)
0927 11:08:39.761417 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:39.761423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:39.761465 (+ 42us) server_negotiation.cc:407] Connection header received
0927 11:08:39.761531 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:39.761532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:39.761543 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:39.761596 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:39.761871 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.762358 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.763156 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.763355 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.763455 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:39.766102 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:40809: BlockingRecv error: recv got EOF from 127.17.95.194:40809 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 27 11:08:39 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:39 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:39.876708 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:39.872229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:37321)
0927 11:08:39.872343 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:39.872354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:39.872413 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:39.872516 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:39.872517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:39.872675 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:39.872724 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:39.872725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.873417 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.873419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:39.873935 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:39.873937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.874295 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.874296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:39.874350 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:39.874769 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:39.874773 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:39.876614 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
W20260927 11:08:39.876749 26938 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:39.872293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:37321 (local address 127.17.95.254:46305)
0927 11:08:39.872353 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:39.872355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:39.872397 (+ 42us) server_negotiation.cc:407] Connection header received
0927 11:08:39.872453 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:39.872454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:39.872466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:39.872505 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:39.872776 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.873341 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.874026 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:39.874216 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:39.874297 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:39.876688 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:37321: BlockingRecv error: recv got EOF from 127.17.95.195:37321 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 11:08:40 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:40 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:40.642381 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:40.637816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:42713)
0927 11:08:40.638002 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:40.638014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:40.638077 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:40.638202 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:40.638202 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:40.638354 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:40.638421 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:40.638422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.639119 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.639121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:40.639622 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:40.639624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.639991 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.639992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:40.640050 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:40.640558 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:40.640563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:40.642291 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
W20260927 11:08:40.642439 26942 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:40.637910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:42713 (local address 127.17.95.254:46305)
0927 11:08:40.638015 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:40.638017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:40.638064 (+ 47us) server_negotiation.cc:407] Connection header received
0927 11:08:40.638121 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:40.638121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:40.638131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:40.638185 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:40.638505 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.638991 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.639705 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.639904 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.639974 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:40.642367 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:42713: BlockingRecv error: recv got EOF from 127.17.95.193:42713 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":49,"threads_started":1}
Sep 27 11:08:40 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:40 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:40.770866 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:40.766533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:58981)
0927 11:08:40.766672 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:40.766687 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:40.766732 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:40.766953 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:40.766953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:40.767140 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:40.767187 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:40.767188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.767831 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.767832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:40.768344 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:40.768345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.768679 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.768680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:40.768718 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:40.769167 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:40.769171 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:40.770803 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":52,"threads_started":1}
W20260927 11:08:40.770951 26942 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:40.766670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:58981 (local address 127.17.95.254:46305)
0927 11:08:40.766834 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:40.766836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:40.766843 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:40.766863 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:40.766863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:40.766873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:40.766916 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:40.767261 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.767767 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.768419 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.768619 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.768668 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:40.770892 (+ 2224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:58981: BlockingRecv error: recv got EOF from 127.17.95.194:58981 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Sep 27 11:08:40 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:40 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:40.881950 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:40.877150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:41309)
0927 11:08:40.877321 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:40.877332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:40.877419 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:40.877605 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:40.877606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:40.877793 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:40.877838 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:40.877840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.878824 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.878826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:40.879405 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:40.879407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.879773 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.879774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:40.879832 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:40.880229 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:40.880233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:40.881858 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
W20260927 11:08:40.882009 26942 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:40.877219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:41309 (local address 127.17.95.254:46305)
0927 11:08:40.877306 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:40.877309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:40.877400 (+ 91us) server_negotiation.cc:407] Connection header received
0927 11:08:40.877496 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:40.877497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:40.877517 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:40.877598 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:40.877916 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.878731 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.879503 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:40.879697 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:40.879773 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:40.881937 (+ 2164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:41309: BlockingRecv error: recv got EOF from 127.17.95.195:41309 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 11:08:41 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:41 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:41.651849 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:41.642890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:44631)
0927 11:08:41.643060 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:41.643072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:41.643145 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:41.643357 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:41.643358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:41.643608 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:41.643672 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:41.643675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.644546 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.644547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:41.645071 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:41.645072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.645393 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.645394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:41.645432 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:41.649848 (+ 4416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:41.649853 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:41.651792 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
W20260927 11:08:41.651976 26946 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:41.642972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:44631 (local address 127.17.95.254:46305)
0927 11:08:41.643129 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:41.643131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:41.643142 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:08:41.643200 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:41.643201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:41.643231 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:41.643320 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:41.643760 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.644450 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.645140 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.645330 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.645385 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:41.651883 (+ 6498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:44631: BlockingRecv error: recv got EOF from 127.17.95.193:44631 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
Sep 27 11:08:41 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:41 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:41.775977 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:41.771296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:35949)
0927 11:08:41.771437 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:41.771452 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:41.771503 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:41.771652 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:41.771653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:41.771843 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:41.771891 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:41.771893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.772602 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.772603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:41.773134 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:41.773136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.773487 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.773488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:41.773530 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:41.773977 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:41.773981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:41.775883 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260927 11:08:41.776026 26946 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:41.771366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:35949 (local address 127.17.95.254:46305)
0927 11:08:41.771431 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:41.771433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:41.771488 (+ 55us) server_negotiation.cc:407] Connection header received
0927 11:08:41.771562 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:41.771562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:41.771573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:41.771635 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:41.771950 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.772530 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.773221 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.773411 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.773483 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:41.775956 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:35949: BlockingRecv error: recv got EOF from 127.17.95.194:35949 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 27 11:08:41 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:41 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:41.887153 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:41.882449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:46091)
0927 11:08:41.882641 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:41.882651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:41.882709 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:41.882886 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:41.882887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:41.883038 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:41.883109 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:41.883111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.883824 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.883825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:41.884328 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:41.884329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.884683 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.884684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:41.884723 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:41.885161 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:41.885165 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:41.887044 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
W20260927 11:08:41.887234 26946 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:41.882538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:46091 (local address 127.17.95.254:46305)
0927 11:08:41.882605 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:41.882608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:41.882731 (+ 123us) server_negotiation.cc:407] Connection header received
0927 11:08:41.882758 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:41.882759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:41.882773 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:41.882831 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:41.883193 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.883723 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.884426 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:41.884616 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:41.884673 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:41.887137 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:46091: BlockingRecv error: recv got EOF from 127.17.95.195:46091 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 27 11:08:42 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:42 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:42.656771 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:42.652388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:53805)
0927 11:08:42.652535 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:42.652548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:42.652618 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:42.652743 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:42.652744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:42.652893 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:42.652942 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:42.652943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.653631 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.653632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:42.654168 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:42.654169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.654538 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.654540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:42.654591 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:42.655029 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:42.655034 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:42.656715 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
W20260927 11:08:42.656924 26950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:42.652448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:53805 (local address 127.17.95.254:46305)
0927 11:08:42.652628 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:42.652629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:42.652637 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:42.652658 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:42.652659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:42.652668 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:42.652724 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:42.653005 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.653526 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.654241 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.654434 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.654507 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:42.656830 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:53805: BlockingRecv error: recv got EOF from 127.17.95.193:53805 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":123,"threads_started":1}
Sep 27 11:08:42 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:42 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:42.781194 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:42.776447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:45897)
0927 11:08:42.776576 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:42.776587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:42.776646 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:42.776750 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:42.776751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:42.776894 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:42.776939 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:42.776941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.777563 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.777564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:42.778081 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:42.778082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.778500 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.778501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:42.778579 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:42.779125 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:42.779133 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:42.781139 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
W20260927 11:08:42.781333 26950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:42.776471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:45897 (local address 127.17.95.254:46305)
0927 11:08:42.776528 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:42.776530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:42.776640 (+ 110us) server_negotiation.cc:407] Connection header received
0927 11:08:42.776694 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:42.776695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:42.776705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:42.776746 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:42.776998 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.777458 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.778173 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.778434 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.778501 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:42.781256 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:45897: BlockingRecv error: recv got EOF from 127.17.95.194:45897 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 27 11:08:42 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:42 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:42.892297 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:42.887664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:47565)
0927 11:08:42.887831 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:42.887842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:42.887905 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:42.888025 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:42.888026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:42.888182 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:42.888230 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:42.888232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.888899 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.888901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:42.889524 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:42.889527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.889856 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.889857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:42.889901 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:42.890468 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:42.890474 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:42.892241 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
W20260927 11:08:42.892390 26950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:42.887724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:47565 (local address 127.17.95.254:46305)
0927 11:08:42.887813 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:42.887815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:42.887907 (+ 92us) server_negotiation.cc:407] Connection header received
0927 11:08:42.887946 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:42.887946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:42.887957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:42.888006 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:42.888299 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.888805 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.889610 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:42.889787 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:42.889838 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:42.892323 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:47565: BlockingRecv error: recv got EOF from 127.17.95.195:47565 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 27 11:08:43 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:43 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:43.661727 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:43.657211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:54515)
0927 11:08:43.657362 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:43.657379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:43.657424 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:43.657602 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:43.657603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:43.657756 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:43.657806 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:43.657807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.658473 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.658474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:43.659024 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:43.659025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.659435 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.659436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:43.659475 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:43.659932 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:43.659936 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:43.661663 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
W20260927 11:08:43.661880 26954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:43.657339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:54515 (local address 127.17.95.254:46305)
0927 11:08:43.657459 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:43.657461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:43.657469 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:43.657512 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:43.657513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:43.657522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:43.657580 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:43.657891 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.658400 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.659172 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.659359 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.659435 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:43.661786 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:54515: BlockingRecv error: recv got EOF from 127.17.95.193:54515 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Sep 27 11:08:43 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:43 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:43.786671 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:43.781682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:34357)
0927 11:08:43.781859 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:43.781869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:43.781925 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:43.782039 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:43.782039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:43.782186 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:43.782230 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:43.782231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.783332 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.783334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:43.783928 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:43.783930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.784335 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.784336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:43.784385 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:43.784896 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:43.784900 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:43.786605 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
W20260927 11:08:43.786795 26954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:43.781732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:34357 (local address 127.17.95.254:46305)
0927 11:08:43.781811 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:43.781813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:43.781928 (+ 115us) server_negotiation.cc:407] Connection header received
0927 11:08:43.781964 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:43.781964 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:43.781979 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:43.782026 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:43.782339 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.783255 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.784027 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.784248 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.784312 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:43.786718 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:34357: BlockingRecv error: recv got EOF from 127.17.95.194:34357 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 11:08:43 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:43 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:43.897033 26956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:43.892772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:55395)
0927 11:08:43.892960 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:43.892969 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:43.893032 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:43.893156 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:43.893157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:43.893321 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:43.893368 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:43.893369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.894021 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.894022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:43.894482 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:43.894484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.894769 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.894770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:43.894811 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:43.895247 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:43.895251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:43.896956 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":107,"threads_started":1}
W20260927 11:08:43.897132 26954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:43.892866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:55395 (local address 127.17.95.254:46305)
0927 11:08:43.892919 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:43.892921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:43.893021 (+ 100us) server_negotiation.cc:407] Connection header received
0927 11:08:43.893085 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:43.893085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:43.893098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:43.893137 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:43.893437 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.893947 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.894550 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:43.894708 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:43.894756 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:43.897047 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:55395: BlockingRecv error: recv got EOF from 127.17.95.195:55395 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 27 11:08:44 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:44 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:44.666692 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:44.662182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:57047)
0927 11:08:44.662360 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:44.662373 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:44.662426 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:44.662565 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:44.662566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:44.662718 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:44.662766 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:44.662768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.663465 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.663466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:44.664008 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:44.664009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.664382 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.664383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:44.664434 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:44.664875 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:44.664879 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:44.666629 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260927 11:08:44.666893 26958 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:44.662253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:57047 (local address 127.17.95.254:46305)
0927 11:08:44.662402 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:44.662404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:44.662425 (+ 21us) server_negotiation.cc:407] Connection header received
0927 11:08:44.662486 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:44.662487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:44.662496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:44.662544 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:44.662829 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.663372 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.664094 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.664289 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.664346 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:44.666794 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:57047: BlockingRecv error: recv got EOF from 127.17.95.193:57047 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
Sep 27 11:08:44 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:44 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:44.791476 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:44.787119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:45999)
0927 11:08:44.787243 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:44.787254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:44.787343 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:44.787464 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:44.787465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:44.787633 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:44.787681 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:44.787683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.788334 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.788335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:44.788859 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:44.788860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.789179 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.789180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:44.789217 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:44.789635 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:44.789639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:44.791421 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260927 11:08:44.791608 26958 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:44.787195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:45999 (local address 127.17.95.254:46305)
0927 11:08:44.787287 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:44.787289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:44.787328 (+ 39us) server_negotiation.cc:407] Connection header received
0927 11:08:44.787386 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:44.787386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:44.787397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:44.787446 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:44.787742 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.788262 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.788927 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.789121 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.789168 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:44.791532 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:45999: BlockingRecv error: recv got EOF from 127.17.95.194:45999 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 11:08:44 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:44 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:44.902227 26960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:44.897479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:33123)
0927 11:08:44.897653 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:44.897666 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:44.897722 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:44.897854 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:44.897854 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:44.898046 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:44.898092 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:44.898094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.898960 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.898961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:44.899566 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:44.899568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.899923 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.899923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:44.899979 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:44.900447 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:44.900452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:44.902141 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
W20260927 11:08:44.902272 26958 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:44.897583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:33123 (local address 127.17.95.254:46305)
0927 11:08:44.897655 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:44.897658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:44.897710 (+ 52us) server_negotiation.cc:407] Connection header received
0927 11:08:44.897780 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:44.897781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:44.897794 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:44.897849 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:44.898173 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.898879 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.899655 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:44.899847 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:44.899913 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:44.902213 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:33123: BlockingRecv error: recv got EOF from 127.17.95.195:33123 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 27 11:08:45 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:45 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:45.671800 26961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:45.667282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:40227)
0927 11:08:45.667401 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:45.667414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:45.667506 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:45.667695 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:45.667696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:45.667856 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:45.667903 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:45.667904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.668513 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.668514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:45.669138 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:45.669141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.669473 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.669474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:45.669518 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:45.669934 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:45.669938 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:45.671701 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
W20260927 11:08:45.671859 26962 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:45.667364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:40227 (local address 127.17.95.254:46305)
0927 11:08:45.667511 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:45.667512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:45.667536 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:08:45.667571 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:45.667571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:45.667581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:45.667628 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:45.667980 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.668447 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.669215 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.669409 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.669459 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:45.671790 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:40227: BlockingRecv error: recv got EOF from 127.17.95.193:40227 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
Sep 27 11:08:45 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:45 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:45.795969 26963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:45.791912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:33959)
0927 11:08:45.792089 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:45.792099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:45.792155 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:45.792277 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:45.792278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:45.792426 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:45.792475 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:45.792476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.793045 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.793045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:45.793508 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:45.793509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.793795 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.793796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:45.793837 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:45.794206 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:45.794210 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:45.795914 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":111,"threads_started":1}
W20260927 11:08:45.796095 26962 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:45.791996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:33959 (local address 127.17.95.254:46305)
0927 11:08:45.792064 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:45.792066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:45.792157 (+ 91us) server_negotiation.cc:407] Connection header received
0927 11:08:45.792197 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:45.792198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:45.792207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:45.792261 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:45.792544 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.792983 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.793569 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.793733 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.793778 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:45.796017 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:33959: BlockingRecv error: recv got EOF from 127.17.95.194:33959 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 27 11:08:45 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:45 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:45.907232 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:45.902726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:38209)
0927 11:08:45.902849 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:45.902860 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:45.902906 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:45.903051 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:45.903052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:45.903278 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:45.903336 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:45.903338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.904061 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.904063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:45.904612 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:45.904614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.904982 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.904990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:45.905058 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:45.905455 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:45.905459 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:45.907130 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
W20260927 11:08:45.907259 26962 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:45.902834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:38209 (local address 127.17.95.254:46305)
0927 11:08:45.902882 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:45.902884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:45.902905 (+ 21us) server_negotiation.cc:407] Connection header received
0927 11:08:45.902965 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:45.902966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:45.902979 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:45.903044 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:45.903406 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.903964 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.904711 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:45.904900 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:45.904979 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:45.907191 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:38209: BlockingRecv error: recv got EOF from 127.17.95.195:38209 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 27 11:08:46 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:46 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:46.680150 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:46.672313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:56361)
0927 11:08:46.672450 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:46.672461 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:46.672516 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:46.672645 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:46.672646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:46.672801 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:46.672846 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:46.672847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.673534 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.673535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:46.674083 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:46.674084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.674377 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.674377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:46.674412 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:46.678137 (+ 3725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:46.678141 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:46.680090 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
W20260927 11:08:46.680265 26966 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:46.672336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:56361 (local address 127.17.95.254:46305)
0927 11:08:46.672471 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:46.672473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:46.672500 (+ 27us) server_negotiation.cc:407] Connection header received
0927 11:08:46.672565 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:46.672565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:46.672575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:46.672633 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:46.672904 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.673445 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.674155 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.674323 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.674367 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:46.680171 (+ 5804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:56361: BlockingRecv error: recv got EOF from 127.17.95.193:56361 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
Sep 27 11:08:46 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:46 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:46.800882 26967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:46.796434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:32927)
0927 11:08:46.796571 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:46.796582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:46.796636 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:46.796743 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:46.796743 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:46.796900 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:46.796944 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:46.796945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.797569 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.797570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:46.798118 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:46.798130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.798443 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.798444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:46.798483 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:46.799004 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:46.799009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:46.800780 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
W20260927 11:08:46.800920 26966 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:46.796520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:32927 (local address 127.17.95.254:46305)
0927 11:08:46.796599 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:46.796601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:46.796622 (+ 21us) server_negotiation.cc:407] Connection header received
0927 11:08:46.796687 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:46.796688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:46.796698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:46.796732 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:46.797012 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.797484 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.798205 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.798384 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.798432 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:46.800859 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:32927: BlockingRecv error: recv got EOF from 127.17.95.194:32927 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 11:08:46 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:46 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:46.912726 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:46.907718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:60961)
0927 11:08:46.907892 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:46.907907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:46.908005 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:46.908201 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:46.908202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:46.908393 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:46.908441 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:46.908442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.909145 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.909146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:46.909693 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:46.909695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.910084 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.910085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:46.910146 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:46.910620 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:46.910624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:46.912605 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
W20260927 11:08:46.912796 26966 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:46.907816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:60961 (local address 127.17.95.254:46305)
0927 11:08:46.907897 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:46.907900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:46.907984 (+ 84us) server_negotiation.cc:407] Connection header received
0927 11:08:46.908097 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:46.908098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:46.908115 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:46.908195 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:46.908534 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.909040 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.909803 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:46.909985 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:46.910072 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:46.912724 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:60961: BlockingRecv error: recv got EOF from 127.17.95.195:60961 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 11:08:47 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:47 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:47.684787 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:47.680605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:46285)
0927 11:08:47.680744 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:47.680754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:47.680800 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:47.680943 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:47.680943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:47.681122 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:47.681167 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:47.681168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.681748 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.681748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:47.682201 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:47.682203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.682488 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.682488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:47.682530 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:47.682963 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:47.682968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:47.684730 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
W20260927 11:08:47.684933 26970 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:47.680679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:46285 (local address 127.17.95.254:46305)
0927 11:08:47.680829 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:47.680830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:47.680838 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:08:47.680857 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:47.680857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:47.680867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:47.680927 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:47.681227 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.681688 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.682265 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.682431 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.682478 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:47.684846 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:46285: BlockingRecv error: recv got EOF from 127.17.95.193:46285 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
Sep 27 11:08:47 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:47 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:47.806438 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:47.801372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:59759)
0927 11:08:47.801537 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:47.801556 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:47.801615 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:47.801752 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:47.801753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:47.801933 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:47.801979 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:47.801980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.802805 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.802806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:47.803423 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:47.803425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.803829 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.803829 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:47.803905 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:47.804354 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:47.804359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:47.806345 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
W20260927 11:08:47.806485 26970 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:47.801477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:59759 (local address 127.17.95.254:46305)
0927 11:08:47.801559 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:47.801561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:47.801613 (+ 52us) server_negotiation.cc:407] Connection header received
0927 11:08:47.801671 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:47.801672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:47.801683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:47.801723 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:47.802056 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.802600 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.803513 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.803756 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.803813 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:47.806424 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:59759: BlockingRecv error: recv got EOF from 127.17.95.194:59759 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 11:08:47 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:47 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:47.917884 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:47.913337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:60121)
0927 11:08:47.913514 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:47.913527 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:47.913599 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:47.913743 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:47.913744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:47.913921 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:47.913964 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:47.913965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.914619 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.914620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:47.915157 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:47.915159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.915531 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.915532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:47.915590 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:47.916033 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:47.916037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:47.917828 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
W20260927 11:08:47.918009 26970 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:47.913439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:60121 (local address 127.17.95.254:46305)
0927 11:08:47.913516 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:47.913518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:47.913592 (+ 74us) server_negotiation.cc:407] Connection header received
0927 11:08:47.913652 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:47.913653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:47.913666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:47.913739 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:47.914050 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.914540 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.915240 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:47.915432 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:47.915531 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:47.917938 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:60121: BlockingRecv error: recv got EOF from 127.17.95.195:60121 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 11:08:48 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:48 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:48.689584 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:48.685210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:58539)
0927 11:08:48.685321 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:48.685333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:48.685401 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:48.685529 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:48.685529 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:48.685689 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:48.685745 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:48.685746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.686361 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.686361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:48.686889 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:48.686890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.687307 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.687307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:48.687348 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:48.687750 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:48.687754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:48.689522 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
W20260927 11:08:48.689700 26974 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:48.685279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:58539 (local address 127.17.95.254:46305)
0927 11:08:48.685398 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:48.685399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:48.685408 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:08:48.685452 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:48.685452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:48.685463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:48.685514 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:48.685813 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.686297 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.686973 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.687237 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.687298 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:48.689619 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:58539: BlockingRecv error: recv got EOF from 127.17.95.193:58539 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
Sep 27 11:08:48 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:48 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:48.811681 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:48.806904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:54575)
0927 11:08:48.807038 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:48.807068 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:48.807159 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:48.807315 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:48.807316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:48.807500 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:48.807544 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:48.807545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.808212 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.808214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:48.808768 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:48.808770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.809134 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.809136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:48.809200 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:48.809671 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:48.809676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:48.811614 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":95,"threads_started":1}
W20260927 11:08:48.811787 26974 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:48.807027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:54575 (local address 127.17.95.254:46305)
0927 11:08:48.807114 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:48.807116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:48.807141 (+ 25us) server_negotiation.cc:407] Connection header received
0927 11:08:48.807247 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:48.807247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:48.807260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:48.807311 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:48.807620 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.808118 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.808852 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.809060 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.809136 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:48.811696 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:54575: BlockingRecv error: recv got EOF from 127.17.95.194:54575 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 11:08:48 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:48 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:48.922950 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:48.918346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:34957)
0927 11:08:48.918501 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:48.918511 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:48.918571 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:48.918685 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:48.918686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:48.918843 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:48.918887 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:48.918889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.919517 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.919517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:48.920084 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:48.920086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.920488 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.920489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:48.920545 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:48.921094 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:48.921098 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:48.922874 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
W20260927 11:08:48.923156 26974 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:48.918441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:34957 (local address 127.17.95.254:46305)
0927 11:08:48.918515 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:48.918517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:48.918569 (+ 52us) server_negotiation.cc:407] Connection header received
0927 11:08:48.918613 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:48.918614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:48.918625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:48.918669 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:48.918951 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.919451 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.920183 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:48.920394 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:48.920466 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:48.923042 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:34957: BlockingRecv error: recv got EOF from 127.17.95.195:34957 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 27 11:08:49 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:49 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:49.694586 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:49.690108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:41979)
0927 11:08:49.690262 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:49.690272 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:49.690378 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:49.690558 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:49.690559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:49.690717 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:49.690764 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:49.690765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.691391 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.691392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:49.691924 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:49.691925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.692241 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.692242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:49.692279 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:49.692774 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:49.692778 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:49.694530 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
W20260927 11:08:49.694653 26978 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:49.690162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:41979 (local address 127.17.95.254:46305)
0927 11:08:49.690267 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:49.690269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:49.690341 (+ 72us) server_negotiation.cc:407] Connection header received
0927 11:08:49.690464 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:49.690464 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:49.690477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:49.690548 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:49.690824 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.691328 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.691993 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.692183 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.692231 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:49.694602 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:41979: BlockingRecv error: recv got EOF from 127.17.95.193:41979 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":45,"threads_started":1}
Sep 27 11:08:49 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:49 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:49.816725 26979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:49.812116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:56459)
0927 11:08:49.812315 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:49.812327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:49.812415 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:49.812575 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:49.812576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:49.812761 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:49.812805 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:49.812806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.813455 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.813456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:49.813967 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:49.813969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.814370 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.814371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:49.814414 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:49.814919 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:49.814923 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:49.816635 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
W20260927 11:08:49.816774 26978 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:49.812199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:56459 (local address 127.17.95.254:46305)
0927 11:08:49.812293 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:49.812295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:49.812396 (+ 101us) server_negotiation.cc:407] Connection header received
0927 11:08:49.812482 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:49.812483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:49.812495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:49.812570 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:49.812894 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.813372 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.814108 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.814304 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.814365 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:49.816713 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:56459: BlockingRecv error: recv got EOF from 127.17.95.194:56459 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 11:08:49 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:49 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.195@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:49.928169 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:49.923423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.195:39337)
0927 11:08:49.923558 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:49.923569 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:49.923633 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:49.923762 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:49.923763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:49.923921 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:49.923966 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:49.923968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.924635 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.924636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:49.925171 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:49.925173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.925495 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.925496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:49.925539 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:49.926011 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:49.926017 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:49.928108 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":55,"threads_started":1}
W20260927 11:08:49.928228 26978 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:49.923515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:39337 (local address 127.17.95.254:46305)
0927 11:08:49.923562 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:49.923564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:49.923621 (+ 57us) server_negotiation.cc:407] Connection header received
0927 11:08:49.923676 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:49.923677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:49.923691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:49.923748 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:49.924056 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.924563 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.925244 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:49.925423 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:49.925476 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:49.928183 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.195:39337: BlockingRecv error: recv got EOF from 127.17.95.195:39337 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 27 11:08:50 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:50 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.193@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:50.699530 26981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:50.695109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.193:46527)
0927 11:08:50.695248 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:50.695257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:50.695314 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:50.695429 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:50.695430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:50.695610 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:50.695658 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:50.695659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.696273 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.696280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:50.696792 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:50.696793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.697097 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.697098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:50.697135 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:50.697627 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:50.697631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:50.699458 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260927 11:08:50.699751 26982 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:50.695180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:46527 (local address 127.17.95.254:46305)
0927 11:08:50.695271 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:50.695273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:50.695303 (+ 30us) server_negotiation.cc:407] Connection header received
0927 11:08:50.695353 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:50.695353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:50.695363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:50.695414 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:50.695724 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.696209 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.696861 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.697039 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.697085 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:50.699684 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.193:46527: BlockingRecv error: recv got EOF from 127.17.95.193:46527 (error 108)
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":44,"threads_started":1}
Sep 27 11:08:50 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 11:08:50 dist-test-slave-6j0v krb5kdc[25923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.95.194@KRBTEST.COM for oryx/127.17.95.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 11:08:50.822031 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:50.817308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:46305 (local address 127.17.95.194:42981)
0927 11:08:50.817461 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:50.817472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:50.817527 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:50.817677 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:50.817678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:50.817853 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:50.817934 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:50.817936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.818850 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.818850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:50.819485 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:50.819487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.819806 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.819806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:50.819845 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:50.820301 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:50.820304 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:50.821967 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.95.254:46305: Server oryx/127.17.95.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260927 11:08:50.822135 26982 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:50.817420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:42981 (local address 127.17.95.254:46305)
0927 11:08:50.817487 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:50.817489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:50.817519 (+ 30us) server_negotiation.cc:407] Connection header received
0927 11:08:50.817572 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:50.817573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:50.817583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:50.817632 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:50.818075 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.818769 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.819562 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:50.819744 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:50.819798 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:50.822061 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.95.194:42981: BlockingRecv error: recv got EOF from 127.17.95.194:42981 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 11:08:50.850068 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26506
I20260927 11:08:50.854472 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26658
I20260927 11:08:50.858598 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26790
I20260927 11:08:50.862535 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26436
2026-09-27T11:08:50Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17368 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T11:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:50Z Disabled control of system clock
I20260927 11:08:50.889087 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:43535
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:43535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 11:08:50.966441 26987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:50.966624 26987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:50.966641 26987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:50.967798 26987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:50.967837 26987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:50.967850 26987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:50.967862 26987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:50.967896 26987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:50.969228 26987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:43535
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:43535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:50.969447 26987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:50.969710 26987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:50.974445 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:50.974455 26993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:50.974519 26987 server_base.cc:1034] running on GCE node
W20260927 11:08:50.974455 26995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:50.974879 26987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:50.975171 26987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:50.976323 26987 hybrid_clock.cc:648] HybridClock initialized: now 1790507330976309 us; error 29 us; skew 500 ppm
I20260927 11:08:50.977439 26987 webserver.cc:460] Webserver started at http://127.17.95.254:34495/ using document root <none> and password file <none>
I20260927 11:08:50.977699 26987 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:50.977744 26987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:50.977890 26987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:50.978871 26987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "27897516a8314021909c0a97fc27e8b6"
format_stamp: "Formatted at 2026-09-27 11:08:50 on dist-test-slave-6j0v"
server_key: "78e936ca7e225fcece1aefcf612400a3"
server_key_iv: "b1ca5a9cbaa391dcc745261e2e38497d"
server_key_version: "encryptionkey@0"
I20260927 11:08:50.979272 26987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "27897516a8314021909c0a97fc27e8b6"
format_stamp: "Formatted at 2026-09-27 11:08:50 on dist-test-slave-6j0v"
server_key: "78e936ca7e225fcece1aefcf612400a3"
server_key_iv: "b1ca5a9cbaa391dcc745261e2e38497d"
server_key_version: "encryptionkey@0"
I20260927 11:08:50.980510 26987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:08:50.981244 27001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:50.981485 26987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:08:50.981580 26987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "27897516a8314021909c0a97fc27e8b6"
format_stamp: "Formatted at 2026-09-27 11:08:50 on dist-test-slave-6j0v"
server_key: "78e936ca7e225fcece1aefcf612400a3"
server_key_iv: "b1ca5a9cbaa391dcc745261e2e38497d"
server_key_version: "encryptionkey@0"
I20260927 11:08:50.981654 26987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:50.992046 26987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:50.992331 26987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:50.992487 26987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:50.996441 26987 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:43535
I20260927 11:08:50.996472 27053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:43535 every 8 connection(s)
I20260927 11:08:50.996783 26987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:50.997360 27054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:50.999747 27054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6: Bootstrap starting.
I20260927 11:08:51.000294 27054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:51.000588 27054 log.cc:824] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:51.001191 27054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6: No bootstrap required, opened a new log
I20260927 11:08:51.002673 27054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27897516a8314021909c0a97fc27e8b6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43535 } }
I20260927 11:08:51.002838 27054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:51.002877 27054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27897516a8314021909c0a97fc27e8b6, State: Initialized, Role: FOLLOWER
I20260927 11:08:51.002980 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 26987
I20260927 11:08:51.003108 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:51.003052 27054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27897516a8314021909c0a97fc27e8b6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43535 } }
I20260927 11:08:51.003170 27054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:51.003227 27054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:51.003273 27054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:51.003357 17791 external_mini_cluster.cc:1428] Setting key 52c31ce0540875e4e430c5e54b0e2a89
I20260927 11:08:51.004019 27054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27897516a8314021909c0a97fc27e8b6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43535 } }
I20260927 11:08:51.004117 27054 leader_election.cc:302] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27897516a8314021909c0a97fc27e8b6; no voters:
I20260927 11:08:51.004285 27054 leader_election.cc:288] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:51.004352 27059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:51.004518 27059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [term 1 LEADER]: Becoming Leader. State: Replica: 27897516a8314021909c0a97fc27e8b6, State: Running, Role: LEADER
I20260927 11:08:51.004534 27054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:51.004642 27059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27897516a8314021909c0a97fc27e8b6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43535 } }
I20260927 11:08:51.005059 27060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27897516a8314021909c0a97fc27e8b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27897516a8314021909c0a97fc27e8b6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43535 } } }
I20260927 11:08:51.005148 27060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:51.005360 27061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27897516a8314021909c0a97fc27e8b6. Latest consensus state: current_term: 1 leader_uuid: "27897516a8314021909c0a97fc27e8b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27897516a8314021909c0a97fc27e8b6" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43535 } } }
I20260927 11:08:51.005638 27065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:51.006212 27065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:51.006881 27061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:51.007860 27065 catalog_manager.cc:1347] Generated new cluster ID: eb96df26efd6442da6a2d28e6c8685f7
I20260927 11:08:51.007907 27065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:51.010004 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.003880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.17.95.254:43535)
0927 11:08:51.004111 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.004119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.004150 (+ 31us) server_negotiation.cc:407] Connection header received
0927 11:08:51.004280 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.004286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.004363 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.004731 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.004861 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.006600 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.007705 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.008283 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.008385 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.009227 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.009230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.009230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.009420 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.009457 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.009470 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.009519 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.009733 (+ 214us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.009800 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260927 11:08:51.021726 27065 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:51.022220 27065 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:51.024551 27065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27897516a8314021909c0a97fc27e8b6: Generated new TSK 0
I20260927 11:08:51.024703 27065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:51.030457 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43535
--builtin_ntp_servers=127.17.95.212:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 11:08:51.109364 27078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.109592 27078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.109628 27078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.111338 27078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.111413 27078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.111455 27078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:51.112751 27078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.113121 27078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.113443 27078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.118609 27084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.118724 27086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.118654 27083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.119051 27078 server_base.cc:1034] running on GCE node
I20260927 11:08:51.119287 27078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.119493 27078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.120647 27078 hybrid_clock.cc:648] HybridClock initialized: now 1790507331120631 us; error 40 us; skew 500 ppm
I20260927 11:08:51.121941 27078 webserver.cc:460] Webserver started at http://127.17.95.193:34491/ using document root <none> and password file <none>
I20260927 11:08:51.122213 27078 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.122285 27078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.122440 27078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.123642 27078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "f98191ff59f2471ca451890767a6c221"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "854287035e784105b9150d839f3f1c49"
server_key_iv: "a0c65ffe88acf2931ba207cfdb11bb30"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.124017 27078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "f98191ff59f2471ca451890767a6c221"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "854287035e784105b9150d839f3f1c49"
server_key_iv: "a0c65ffe88acf2931ba207cfdb11bb30"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.125424 27078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:51.126214 27092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.126397 27078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:51.126505 27078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "f98191ff59f2471ca451890767a6c221"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "854287035e784105b9150d839f3f1c49"
server_key_iv: "a0c65ffe88acf2931ba207cfdb11bb30"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.126590 27078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.178387 27078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.178743 27078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.178900 27078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.179198 27078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:51.179561 27078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:51.179618 27078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.179663 27078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:51.179692 27078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.185832 27078 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:39743
I20260927 11:08:51.185895 27205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:39743 every 8 connection(s)
I20260927 11:08:51.186220 27078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:51.191952 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.186888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:57597 (local address 127.17.95.254:43535)
0927 11:08:51.186956 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.186958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.187260 (+ 302us) server_negotiation.cc:407] Connection header received
0927 11:08:51.187918 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.187920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.187935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.187992 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.188744 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.189325 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.190409 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.190633 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.190714 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.191651 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.191652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.191653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.191658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.191672 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.191676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.191732 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.191837 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.191867 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:08:51.192193 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.186886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:43535 (local address 127.17.95.193:57597)
0927 11:08:51.187110 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:51.187153 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:51.187798 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:51.188090 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:51.188102 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:51.188111 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:51.188632 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:51.188637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.189427 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.189428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:51.190294 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:51.190296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.190723 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.190724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:51.190869 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.191484 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:51.191484 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:51.191516 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:51.191539 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:51.191737 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:51.191742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:51.191745 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:51.191841 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 11:08:51.191945 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260927 11:08:51.192710 27206 heartbeater.cc:347] Connected to a master server at 127.17.95.254:43535
I20260927 11:08:51.192803 27206 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:51.192996 27206 heartbeater.cc:511] Master 127.17.95.254:43535 requested a full tablet report, sending...
I20260927 11:08:51.193536 27018 ts_manager.cc:194] Registered new tserver with Master: f98191ff59f2471ca451890767a6c221 (127.17.95.193:39743)
I20260927 11:08:51.194166 27018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.193:57597
I20260927 11:08:51.195804 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27078
I20260927 11:08:51.195881 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:51.196024 17791 external_mini_cluster.cc:1428] Setting key af68ad2974526b2f933f27a9b5153663
I20260927 11:08:51.196787 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43535
--builtin_ntp_servers=127.17.95.212:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 11:08:51.276841 27209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.277043 27209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.277066 27209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.278116 27209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.278154 27209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.278180 27209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:51.279433 27209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.279640 27209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.279870 27209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.285012 27215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.285130 27209 server_base.cc:1034] running on GCE node
W20260927 11:08:51.285027 27214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.285066 27217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.285451 27209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.285735 27209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.286896 27209 hybrid_clock.cc:648] HybridClock initialized: now 1790507331286882 us; error 34 us; skew 500 ppm
I20260927 11:08:51.288194 27209 webserver.cc:460] Webserver started at http://127.17.95.194:33943/ using document root <none> and password file <none>
I20260927 11:08:51.288445 27209 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.288498 27209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.288650 27209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.289685 27209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "68ce8f949ba34b4f842dab62f9444f72"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "6911db08bf8491d75389a533c60fdec1"
server_key_iv: "4a35987d0e2d39b2fa6bacdf7e898afd"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.290036 27209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "68ce8f949ba34b4f842dab62f9444f72"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "6911db08bf8491d75389a533c60fdec1"
server_key_iv: "4a35987d0e2d39b2fa6bacdf7e898afd"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.291376 27209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:51.292145 27223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.292354 27209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:51.292450 27209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "68ce8f949ba34b4f842dab62f9444f72"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "6911db08bf8491d75389a533c60fdec1"
server_key_iv: "4a35987d0e2d39b2fa6bacdf7e898afd"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.292536 27209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.304705 27209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.305034 27209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.305190 27209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.305457 27209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:51.305784 27209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:51.305840 27209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.305882 27209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:51.305910 27209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.311836 27209 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:32811
I20260927 11:08:51.311875 27336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:32811 every 8 connection(s)
I20260927 11:08:51.312218 27209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:51.317535 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.313130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:39253 (local address 127.17.95.254:43535)
0927 11:08:51.313182 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.313184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.313315 (+ 131us) server_negotiation.cc:407] Connection header received
0927 11:08:51.313812 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.313813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.313824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.313894 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.314571 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.315040 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.316108 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.316295 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.316361 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.317297 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.317298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.317298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.317302 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.317317 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.317320 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.317367 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.317441 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.317467 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 11:08:51.317740 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.312930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:43535 (local address 127.17.95.194:39253)
0927 11:08:51.313181 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:51.313228 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:51.313713 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:51.313960 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:51.313963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:51.313972 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:51.314502 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:51.314506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.315174 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.315175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:51.316019 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:51.316021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.316401 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.316402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:51.316526 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.317146 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:51.317146 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:51.317176 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:51.317200 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:51.317375 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:51.317378 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:51.317380 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:51.317431 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 11:08:51.317513 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260927 11:08:51.318246 27337 heartbeater.cc:347] Connected to a master server at 127.17.95.254:43535
I20260927 11:08:51.318326 27337 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:51.318476 27337 heartbeater.cc:511] Master 127.17.95.254:43535 requested a full tablet report, sending...
I20260927 11:08:51.318805 27018 ts_manager.cc:194] Registered new tserver with Master: 68ce8f949ba34b4f842dab62f9444f72 (127.17.95.194:32811)
I20260927 11:08:51.319273 27018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.194:39253
I20260927 11:08:51.321532 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27209
I20260927 11:08:51.321625 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:51.321820 17791 external_mini_cluster.cc:1428] Setting key 433bf12295aebbfd79a38f19ec25f4eb
I20260927 11:08:51.322854 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43535
--builtin_ntp_servers=127.17.95.212:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 11:08:51.400868 27340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.401093 27340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.401125 27340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.402268 27340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.402333 27340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.402374 27340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:51.403697 27340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.403949 27340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.404217 27340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.409075 27346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.409060 27348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.409080 27345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.409284 27340 server_base.cc:1034] running on GCE node
I20260927 11:08:51.409513 27340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.409758 27340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.410915 27340 hybrid_clock.cc:648] HybridClock initialized: now 1790507331410896 us; error 34 us; skew 500 ppm
I20260927 11:08:51.412161 27340 webserver.cc:460] Webserver started at http://127.17.95.195:46651/ using document root <none> and password file <none>
I20260927 11:08:51.412388 27340 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.412451 27340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.412582 27340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.413524 27340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "626f9945b98e48f8a98044e5f499933f"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "64381ee14bd198ef0fafbcf8cf5297e1"
server_key_iv: "cfebbbf12477065dd01470759586a2ab"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.413861 27340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "626f9945b98e48f8a98044e5f499933f"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "64381ee14bd198ef0fafbcf8cf5297e1"
server_key_iv: "cfebbbf12477065dd01470759586a2ab"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.415112 27340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:51.415858 27354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.416059 27340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:08:51.416152 27340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "626f9945b98e48f8a98044e5f499933f"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "64381ee14bd198ef0fafbcf8cf5297e1"
server_key_iv: "cfebbbf12477065dd01470759586a2ab"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.416234 27340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.427953 27340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.428238 27340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.428387 27340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.428612 27340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:51.428921 27340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:51.428972 27340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.429016 27340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:51.429045 27340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.434702 27340 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:42435
I20260927 11:08:51.434738 27467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:42435 every 8 connection(s)
I20260927 11:08:51.435148 27340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:51.437307 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27340
I20260927 11:08:51.437407 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:51.437592 17791 external_mini_cluster.cc:1428] Setting key 4e1234cb61fbb2c5258596d2e578bdcb
I20260927 11:08:51.441191 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.435822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:48103 (local address 127.17.95.254:43535)
0927 11:08:51.435885 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.435887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.436137 (+ 250us) server_negotiation.cc:407] Connection header received
0927 11:08:51.436618 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.436620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.436634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.436701 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.437499 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.438249 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.439487 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.439810 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.439903 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.440878 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.440880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.440880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.440884 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.440899 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.440903 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.440985 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.441061 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.441088 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 11:08:51.441370 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.435796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:43535 (local address 127.17.95.195:48103)
0927 11:08:51.436016 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:51.436055 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:51.436512 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:51.436766 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:51.436769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:51.436777 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:51.437350 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:51.437357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.438336 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.438337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:51.439392 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:51.439395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:51.439912 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:51.439913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:51.440086 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:51.440713 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:51.440714 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:51.440750 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:51.440779 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:51.440989 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:51.440994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:51.440996 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:51.441063 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 11:08:51.441166 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260927 11:08:51.441792 27468 heartbeater.cc:347] Connected to a master server at 127.17.95.254:43535
I20260927 11:08:51.441885 27468 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:51.442057 27468 heartbeater.cc:511] Master 127.17.95.254:43535 requested a full tablet report, sending...
I20260927 11:08:51.442384 27018 ts_manager.cc:194] Registered new tserver with Master: 626f9945b98e48f8a98044e5f499933f (127.17.95.195:42435)
I20260927 11:08:51.442739 27018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.195:48103
I20260927 11:08:51.449949 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 11:08:51.451680 27058 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:51.451249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.17.95.254:43535)
0927 11:08:51.451309 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.451311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.451400 (+ 89us) server_negotiation.cc:407] Connection header received
0927 11:08:51.451434 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.451434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.451445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.451487 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.451615 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49128: BlockingRecv error: recv got EOF from 127.0.0.1:49128 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:08:51.451716 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27078
I20260927 11:08:51.456317 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27209
I20260927 11:08:51.460623 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27340
I20260927 11:08:51.464563 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 26987
2026-09-27T11:08:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (601 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T11:08:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:51Z Disabled control of system clock
I20260927 11:08:51.490553 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:43469
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 11:08:51.568034 27479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.568224 27479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.568245 27479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.569371 27479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:51.569411 27479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.569425 27479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.569437 27479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:51.569449 27479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:51.570729 27479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:43469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.570962 27479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.571261 27479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.575747 27484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.575737 27485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.575814 27479 server_base.cc:1034] running on GCE node
W20260927 11:08:51.575736 27487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.576124 27479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.576386 27479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.577534 27479 hybrid_clock.cc:648] HybridClock initialized: now 1790507331577522 us; error 28 us; skew 500 ppm
I20260927 11:08:51.578730 27479 webserver.cc:460] Webserver started at http://127.17.95.254:38499/ using document root <none> and password file <none>
I20260927 11:08:51.578962 27479 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.579010 27479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.579159 27479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.580121 27479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "19b517915a4b4cf5863ccb586c29880c"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "3cbcb9f20cdc5a89d1a336d5cec07903"
server_key_iv: "5ac0aac3de4157d96de4faafca5197d6"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.580441 27479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "19b517915a4b4cf5863ccb586c29880c"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "3cbcb9f20cdc5a89d1a336d5cec07903"
server_key_iv: "5ac0aac3de4157d96de4faafca5197d6"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.581647 27479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:51.582320 27493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.582480 27479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:51.582574 27479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "19b517915a4b4cf5863ccb586c29880c"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "3cbcb9f20cdc5a89d1a336d5cec07903"
server_key_iv: "5ac0aac3de4157d96de4faafca5197d6"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.582644 27479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.585867 27479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.586110 27479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.586237 27479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.590025 27479 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:43469
I20260927 11:08:51.590060 27545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:43469 every 8 connection(s)
I20260927 11:08:51.590358 27479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:51.590970 27546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:51.593535 27546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c: Bootstrap starting.
I20260927 11:08:51.594030 27546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:51.594141 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27479
I20260927 11:08:51.594244 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:51.594345 27546 log.cc:824] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:51.594395 17791 external_mini_cluster.cc:1428] Setting key 169693d826f670a3fb891cffe4ea5329
I20260927 11:08:51.595140 27546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c: No bootstrap required, opened a new log
I20260927 11:08:51.596899 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.594844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.17.95.254:43469)
0927 11:08:51.595044 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.595052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.595113 (+ 61us) server_negotiation.cc:407] Connection header received
0927 11:08:51.595586 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.595594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.595646 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.595794 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.596463 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.596466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.596467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.596472 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.596518 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.596531 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.596627 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.596659 (+ 32us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.596710 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260927 11:08:51.597256 27546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b517915a4b4cf5863ccb586c29880c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:08:51.597429 27546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:51.597477 27546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b517915a4b4cf5863ccb586c29880c, State: Initialized, Role: FOLLOWER
I20260927 11:08:51.597600 27546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b517915a4b4cf5863ccb586c29880c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:08:51.597697 27546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:51.597728 27546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:51.597764 27546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:51.598498 27546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b517915a4b4cf5863ccb586c29880c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:08:51.598608 27546 leader_election.cc:302] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19b517915a4b4cf5863ccb586c29880c; no voters:
I20260927 11:08:51.598722 27546 leader_election.cc:288] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:51.598770 27551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:51.598927 27551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [term 1 LEADER]: Becoming Leader. State: Replica: 19b517915a4b4cf5863ccb586c29880c, State: Running, Role: LEADER
I20260927 11:08:51.599025 27546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:51.599020 27551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b517915a4b4cf5863ccb586c29880c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:08:51.599419 27552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19b517915a4b4cf5863ccb586c29880c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b517915a4b4cf5863ccb586c29880c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } } }
I20260927 11:08:51.599423 27553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19b517915a4b4cf5863ccb586c29880c. Latest consensus state: current_term: 1 leader_uuid: "19b517915a4b4cf5863ccb586c29880c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b517915a4b4cf5863ccb586c29880c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } } }
I20260927 11:08:51.599534 27552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:51.599540 27553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:51.599946 27556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:51.600463 27556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:51.601974 27556 catalog_manager.cc:1347] Generated new cluster ID: bb045ac10b194b989be1d5b44bcb5e25
I20260927 11:08:51.602026 27556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:08:51.616155 27556 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:51.616317 27556 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:51.618613 27556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19b517915a4b4cf5863ccb586c29880c: Generated new TSK 0
I20260927 11:08:51.618762 27556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:08:51.625077 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43469
--builtin_ntp_servers=127.17.95.212:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 11:08:51.704480 27570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.704676 27570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.704700 27570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.705816 27570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.705854 27570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.705881 27570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:51.707206 27570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.707499 27570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.707791 27570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.713431 27575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.713502 27576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.713657 27570 server_base.cc:1034] running on GCE node
W20260927 11:08:51.713734 27578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.713954 27570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.714226 27570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.715385 27570 hybrid_clock.cc:648] HybridClock initialized: now 1790507331715364 us; error 42 us; skew 500 ppm
I20260927 11:08:51.722168 27570 webserver.cc:460] Webserver started at http://127.17.95.193:37547/ using document root <none> and password file <none>
I20260927 11:08:51.722422 27570 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.722479 27570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.722605 27570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.723733 27570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "5d8f44db50704a31bbeec9242c0e69fc"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "721f8a39ab5eba6209455cb29f841c71"
server_key_iv: "b296dad5a1f83eb66bc2a7df0859adc1"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.724084 27570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "5d8f44db50704a31bbeec9242c0e69fc"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "721f8a39ab5eba6209455cb29f841c71"
server_key_iv: "b296dad5a1f83eb66bc2a7df0859adc1"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.725324 27570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:08:51.726042 27584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.726207 27570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:51.726306 27570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "5d8f44db50704a31bbeec9242c0e69fc"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "721f8a39ab5eba6209455cb29f841c71"
server_key_iv: "b296dad5a1f83eb66bc2a7df0859adc1"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.726374 27570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.729666 27570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.729933 27570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.730064 27570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.730285 27570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:51.730592 27570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:51.730664 27570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.730715 27570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:51.730742 27570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.736384 27570 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:44375
I20260927 11:08:51.736423 27697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:44375 every 8 connection(s)
I20260927 11:08:51.736724 27570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:08:51.739482 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.737405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:35661 (local address 127.17.95.254:43469)
0927 11:08:51.737447 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.737449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.737646 (+ 197us) server_negotiation.cc:407] Connection header received
0927 11:08:51.738077 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.738078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.738083 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.738131 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.739317 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.739318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.739318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.739321 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.739332 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.739336 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.739380 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.739424 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.739435 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:08:51.739691 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.737388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:43469 (local address 127.17.95.193:35661)
0927 11:08:51.737549 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:51.737579 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:51.737983 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:51.738178 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:51.738181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:51.738188 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:51.739193 (+ 1005us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:51.739193 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:51.739220 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:51.739245 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:51.739378 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:51.739380 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:51.739382 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:51.739432 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 11:08:51.739473 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260927 11:08:51.739880 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27570
I20260927 11:08:51.739961 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:51.740152 17791 external_mini_cluster.cc:1428] Setting key 5835a01381749048236f7698b5ae365b
I20260927 11:08:51.740568 27698 heartbeater.cc:347] Connected to a master server at 127.17.95.254:43469
I20260927 11:08:51.740638 27698 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:51.740790 27698 heartbeater.cc:511] Master 127.17.95.254:43469 requested a full tablet report, sending...
I20260927 11:08:51.741253 27509 ts_manager.cc:194] Registered new tserver with Master: 5d8f44db50704a31bbeec9242c0e69fc (127.17.95.193:44375)
I20260927 11:08:51.741283 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43469
--builtin_ntp_servers=127.17.95.212:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 11:08:51.817951 27701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.818140 27701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.818161 27701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.819324 27701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.819361 27701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.819386 27701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:51.820667 27701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.820876 27701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.821148 27701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.825891 27709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.825903 27706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.825917 27707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.826122 27701 server_base.cc:1034] running on GCE node
I20260927 11:08:51.826390 27701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.826663 27701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.827812 27701 hybrid_clock.cc:648] HybridClock initialized: now 1790507331827790 us; error 35 us; skew 500 ppm
I20260927 11:08:51.829197 27701 webserver.cc:460] Webserver started at http://127.17.95.194:45257/ using document root <none> and password file <none>
I20260927 11:08:51.829377 27701 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.829416 27701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.829510 27701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.830507 27701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "490d3744983f445f89b65a25f4c846da"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "fe4b2aa44b10218974f6d093eb7479ff"
server_key_iv: "2047f583cdfdc8699127fef6028ff5c2"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.830787 27701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "490d3744983f445f89b65a25f4c846da"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "fe4b2aa44b10218974f6d093eb7479ff"
server_key_iv: "2047f583cdfdc8699127fef6028ff5c2"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.832019 27701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:08:51.832695 27715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.832872 27701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:51.832957 27701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "490d3744983f445f89b65a25f4c846da"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "fe4b2aa44b10218974f6d093eb7479ff"
server_key_iv: "2047f583cdfdc8699127fef6028ff5c2"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.833031 27701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.836229 27701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.836452 27701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.836567 27701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.836804 27701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:51.837113 27701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:51.837144 27701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.837163 27701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:51.837205 27701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.842577 27701 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:43655
I20260927 11:08:51.842618 27828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:43655 every 8 connection(s)
I20260927 11:08:51.842888 27701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:51.845670 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.843565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:52551 (local address 127.17.95.254:43469)
0927 11:08:51.843644 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.843646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.843844 (+ 198us) server_negotiation.cc:407] Connection header received
0927 11:08:51.844223 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.844224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.844229 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.844278 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.845420 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.845422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.845422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.845426 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.845439 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.845443 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.845508 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.845561 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.845572 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 11:08:51.845804 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27701
I20260927 11:08:51.845816 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.843524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:43469 (local address 127.17.95.194:52551)
0927 11:08:51.843741 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:51.843770 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:51.844150 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:51.844336 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:51.844339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:51.844346 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:51.845277 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:51.845278 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:51.845303 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:51.845328 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:51.845509 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:51.845512 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:51.845514 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:51.845566 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 11:08:51.845624 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260927 11:08:51.845909 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:51.846091 17791 external_mini_cluster.cc:1428] Setting key d461008e613a0ba35edcfab9c15e53d5
I20260927 11:08:51.846336 27829 heartbeater.cc:347] Connected to a master server at 127.17.95.254:43469
I20260927 11:08:51.846428 27829 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:51.846589 27829 heartbeater.cc:511] Master 127.17.95.254:43469 requested a full tablet report, sending...
I20260927 11:08:51.846935 27509 ts_manager.cc:194] Registered new tserver with Master: 490d3744983f445f89b65a25f4c846da (127.17.95.194:43655)
I20260927 11:08:51.847167 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43469
--builtin_ntp_servers=127.17.95.212:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 11:08:51.924324 27832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:51.924533 27832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:51.924567 27832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:51.925657 27832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:51.925714 27832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:51.925755 27832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:51.927009 27832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:51.927291 27832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:51.927559 27832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:51.932348 27840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:51.932330 27838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.932552 27832 server_base.cc:1034] running on GCE node
W20260927 11:08:51.932332 27837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:51.932824 27832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:51.933058 27832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:51.934197 27832 hybrid_clock.cc:648] HybridClock initialized: now 1790507331934186 us; error 23 us; skew 500 ppm
I20260927 11:08:51.935501 27832 webserver.cc:460] Webserver started at http://127.17.95.195:37935/ using document root <none> and password file <none>
I20260927 11:08:51.935738 27832 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:51.935804 27832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:51.935935 27832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:51.937023 27832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "e5c665fefa6d48c4bd87805d8b79e099"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "a88f94e68f9dc9c49f0f2303b435cfd9"
server_key_iv: "e6c6e06ea034afb4de7ee4c4aa4f6a1a"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.937376 27832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "e5c665fefa6d48c4bd87805d8b79e099"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "a88f94e68f9dc9c49f0f2303b435cfd9"
server_key_iv: "e6c6e06ea034afb4de7ee4c4aa4f6a1a"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.938640 27832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:51.939397 27846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.939560 27832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:51.939654 27832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "e5c665fefa6d48c4bd87805d8b79e099"
format_stamp: "Formatted at 2026-09-27 11:08:51 on dist-test-slave-6j0v"
server_key: "a88f94e68f9dc9c49f0f2303b435cfd9"
server_key_iv: "e6c6e06ea034afb4de7ee4c4aa4f6a1a"
server_key_version: "encryptionkey@0"
I20260927 11:08:51.939733 27832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:51.942914 27832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:51.943216 27832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:51.943351 27832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:51.943550 27832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:51.943847 27832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:51.943904 27832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.943940 27832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:51.943966 27832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:51.949306 27832 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:36461
I20260927 11:08:51.949375 27959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:36461 every 8 connection(s)
I20260927 11:08:51.949688 27832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:51.951274 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27832
I20260927 11:08:51.951349 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:51.951543 17791 external_mini_cluster.cc:1428] Setting key 82a5becca5b7e3eeb52509299e1fe5f3
I20260927 11:08:51.952641 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.950372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:47753 (local address 127.17.95.254:43469)
0927 11:08:51.950426 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.950428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.950649 (+ 221us) server_negotiation.cc:407] Connection header received
0927 11:08:51.951048 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.951049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.951055 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.951156 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.952414 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:51.952415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:51.952415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:08:51.952419 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:51.952455 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:08:51.952459 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:51.952517 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:51.952582 (+ 65us) server_negotiation.cc:299] Negotiation successful
0927 11:08:51.952593 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:08:51.952838 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:51.950358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:43469 (local address 127.17.95.195:47753)
0927 11:08:51.950539 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:51.950570 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:51.950962 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:51.951203 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:51.951206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:51.951213 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:51.952279 (+ 1066us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:08:51.952279 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:51.952306 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:08:51.952332 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:51.952525 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:51.952529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:51.952531 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:08:51.952586 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 11:08:51.952630 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260927 11:08:51.953224 27960 heartbeater.cc:347] Connected to a master server at 127.17.95.254:43469
I20260927 11:08:51.953289 27960 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:51.953440 27960 heartbeater.cc:511] Master 127.17.95.254:43469 requested a full tablet report, sending...
I20260927 11:08:51.953738 27509 ts_manager.cc:194] Registered new tserver with Master: e5c665fefa6d48c4bd87805d8b79e099 (127.17.95.195:36461)
I20260927 11:08:51.963382 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 11:08:51.965289 27550 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:08:51.964776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.17.95.254:43469)
0927 11:08:51.964863 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:51.964865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:51.964962 (+ 97us) server_negotiation.cc:407] Connection header received
0927 11:08:51.965036 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:51.965037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:51.965042 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:51.965086 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:51.965224 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54222: BlockingRecv error: recv got EOF from 127.0.0.1:54222 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 11:08:51.965384 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27570
I20260927 11:08:51.970283 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27701
I20260927 11:08:51.974632 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27832
I20260927 11:08:51.978714 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27479
2026-09-27T11:08:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (513 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:52 dist-test-slave-6j0v krb5kdc[27970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:52 dist-test-slave-6j0v krb5kdc[27970](info): set up 2 sockets
Sep 27 11:08:52 dist-test-slave-6j0v krb5kdc[27970](info): commencing operation
krb5kdc: starting...
W20260927 11:08:52.992287 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:08:53.105257 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:37657
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:37657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:53.186705 27983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:53.186941 27983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:53.186990 27983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:53.188186 27983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:08:53.188256 27983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:53.188279 27983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:53.188308 27983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:08:53.188337 27983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:08:53.189800 27983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:37657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:37657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:53.190081 27983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:53.190295 27983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:53.195277 27988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.195262 27991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.195266 27989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:53.195328 27983 server_base.cc:1034] running on GCE node
I20260927 11:08:53.195700 27983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:53.195935 27983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:53.197103 27983 hybrid_clock.cc:648] HybridClock initialized: now 1790507333197091 us; error 52 us; skew 500 ppm
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:53.199561 27983 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:08:53.200007 27983 webserver.cc:460] Webserver started at http://127.17.95.254:37827/ using document root <none> and password file <none>
I20260927 11:08:53.200191 27983 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:53.200232 27983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:53.200315 27983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:53.201274 27983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "333d7a6b925b4a698c52cf98df274276"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "504b10da5bf1b907d1cae6daad7203b7"
server_key_iv: "10ca948e0b4e2065817ac6f3b27dbf87"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.201574 27983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "333d7a6b925b4a698c52cf98df274276"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "504b10da5bf1b907d1cae6daad7203b7"
server_key_iv: "10ca948e0b4e2065817ac6f3b27dbf87"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.202814 27983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:08:53.203532 27998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.203728 27983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.203841 27983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "333d7a6b925b4a698c52cf98df274276"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "504b10da5bf1b907d1cae6daad7203b7"
server_key_iv: "10ca948e0b4e2065817ac6f3b27dbf87"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.203910 27983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:53.215348 27983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:53.217514 27983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:53.217661 27983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:53.221328 27983 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:37657
I20260927 11:08:53.221385 28050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:37657 every 8 connection(s)
I20260927 11:08:53.221695 27983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:08:53.222703 28051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:53.224977 28051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276: Bootstrap starting.
I20260927 11:08:53.225601 28051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:53.225854 28051 log.cc:824] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:53.226529 28051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276: No bootstrap required, opened a new log
I20260927 11:08:53.228098 28051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333d7a6b925b4a698c52cf98df274276" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37657 } }
I20260927 11:08:53.228228 28051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:53.228250 28051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333d7a6b925b4a698c52cf98df274276, State: Initialized, Role: FOLLOWER
I20260927 11:08:53.228344 28051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333d7a6b925b4a698c52cf98df274276" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37657 } }
I20260927 11:08:53.228420 28051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:08:53.228466 28051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:08:53.228549 28051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:53.229349 28051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333d7a6b925b4a698c52cf98df274276" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37657 } }
I20260927 11:08:53.229466 28051 leader_election.cc:302] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 333d7a6b925b4a698c52cf98df274276; no voters:
I20260927 11:08:53.229662 28051 leader_election.cc:288] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:08:53.229683 28054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:53.229676 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 27983
I20260927 11:08:53.229773 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:08:53.229791 28054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [term 1 LEADER]: Becoming Leader. State: Replica: 333d7a6b925b4a698c52cf98df274276, State: Running, Role: LEADER
I20260927 11:08:53.229861 28054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333d7a6b925b4a698c52cf98df274276" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37657 } }
I20260927 11:08:53.229930 28051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:08:53.229990 17791 external_mini_cluster.cc:1428] Setting key 7a613af071db932dfbe0ccf08758299d
I20260927 11:08:53.230163 28054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 333d7a6b925b4a698c52cf98df274276. Latest consensus state: current_term: 1 leader_uuid: "333d7a6b925b4a698c52cf98df274276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333d7a6b925b4a698c52cf98df274276" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37657 } } }
I20260927 11:08:53.230167 28056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "333d7a6b925b4a698c52cf98df274276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333d7a6b925b4a698c52cf98df274276" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37657 } } }
I20260927 11:08:53.230360 28054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:53.230398 28056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276 [sys.catalog]: This master's current role is: LEADER
I20260927 11:08:53.230562 28059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:08:53.231302 28059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:08:53.233042 28059 catalog_manager.cc:1347] Generated new cluster ID: 25d1d5452cfb484e842031bc0962276e
I20260927 11:08:53.233096 28059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:53.238543 28059 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:08:53.239216 28059 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:08:53.240967 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.230626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.17.95.254:37657)
0927 11:08:53.231026 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.231033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.231056 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:08:53.231197 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.231202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.231242 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.231374 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:53.232168 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.233400 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.234163 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.234602 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.234723 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.237044 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:53.237048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:53.237048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:53.237059 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:53.239450 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.239823 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.239826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.239826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.239923 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.240130 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.240131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.240132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.240356 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:53.240534 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.240603 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.240754 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":93,"threads_started":1}
I20260927 11:08:53.241730 28059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276: Generated new TSK 0
I20260927 11:08:53.241906 28059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:08:53.269886 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37657
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:53.358912 28077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:53.359194 28077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:53.359237 28077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:53.360391 28077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:53.360455 28077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:53.360499 28077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:08:53.361901 28077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:53.362170 28077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:53.362396 28077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:08:53.367323 28077 server_base.cc:1034] running on GCE node
W20260927 11:08:53.367288 28085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.367250 28083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.367288 28082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:53.367763 28077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:53.368027 28077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:53.369189 28077 hybrid_clock.cc:648] HybridClock initialized: now 1790507333369175 us; error 31 us; skew 500 ppm
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:53.371897 28077 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:08:53.372465 28077 webserver.cc:460] Webserver started at http://127.17.95.193:35511/ using document root <none> and password file <none>
I20260927 11:08:53.372752 28077 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:53.372810 28077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:53.372936 28077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:53.374245 28077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "70c9f1050d214ddfad97d144c1bf16da"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "33ac965efadcb4e31735748e19e5e397"
server_key_iv: "92c592b6797959c0d0dc87e8ddf1a8cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.374656 28077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "70c9f1050d214ddfad97d144c1bf16da"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "33ac965efadcb4e31735748e19e5e397"
server_key_iv: "92c592b6797959c0d0dc87e8ddf1a8cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.376183 28077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260927 11:08:53.376994 28092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.377193 28077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:08:53.377275 28077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "70c9f1050d214ddfad97d144c1bf16da"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "33ac965efadcb4e31735748e19e5e397"
server_key_iv: "92c592b6797959c0d0dc87e8ddf1a8cc"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.377338 28077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:53.388602 28077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:53.390764 28077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:53.390913 28077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:53.391139 28077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:53.391381 28077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:53.391410 28077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.391430 28077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:53.391443 28077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.397127 28077 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:39203
I20260927 11:08:53.397210 28205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:39203 every 8 connection(s)
I20260927 11:08:53.397501 28077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:53.405216 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 28077
I20260927 11:08:53.405320 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:08:53.405557 17791 external_mini_cluster.cc:1428] Setting key 1986bc74d0f69ec93d1f5ea433cfc9bd
I20260927 11:08:53.407037 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.398185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:56723 (local address 127.17.95.254:37657)
0927 11:08:53.398262 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.398265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.398507 (+ 242us) server_negotiation.cc:407] Connection header received
0927 11:08:53.398976 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.398977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.398991 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.399057 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:53.399948 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.400454 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.401508 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.401728 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.401790 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.403939 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:53.403940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:53.403940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:53.403947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:53.405520 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.405962 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.405963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.405963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.406039 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.406309 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.406309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.406310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.406468 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:53.406559 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.406833 (+ 274us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.406924 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 11:08:53.407204 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.398187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:37657 (local address 127.17.95.193:56723)
0927 11:08:53.398379 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.398412 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.398878 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.399162 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.399165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.399686 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:53.399862 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.399866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.400557 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.400559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.401409 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.401411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.401816 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.401817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.401924 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.402460 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:53.402469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:53.403849 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:53.405610 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:53.405615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:53.405617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:53.405889 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:53.406105 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:53.406106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:53.406106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:53.406241 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:53.406559 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:53.406568 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:53.406681 (+ 113us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.406794 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.406949 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260927 11:08:53.407853 28206 heartbeater.cc:347] Connected to a master server at 127.17.95.254:37657
I20260927 11:08:53.407965 28206 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:53.408136 28206 heartbeater.cc:511] Master 127.17.95.254:37657 requested a full tablet report, sending...
I20260927 11:08:53.408635 28015 ts_manager.cc:194] Registered new tserver with Master: 70c9f1050d214ddfad97d144c1bf16da (127.17.95.193:39203)
I20260927 11:08:53.409313 28015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:56723
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:08:53.432411 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37657
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:53.514263 28211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:53.514443 28211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:53.514465 28211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:53.515604 28211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:53.515643 28211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:53.515669 28211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:08:53.517148 28211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:53.517387 28211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:53.517621 28211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:53.522374 28216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:53.522388 28211 server_base.cc:1034] running on GCE node
W20260927 11:08:53.522390 28219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.522362 28217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:53.522728 28211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:53.522984 28211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:53.524153 28211 hybrid_clock.cc:648] HybridClock initialized: now 1790507333524130 us; error 34 us; skew 500 ppm
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:53.526484 28211 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:08:53.526926 28211 webserver.cc:460] Webserver started at http://127.17.95.194:44835/ using document root <none> and password file <none>
I20260927 11:08:53.527222 28211 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:53.527307 28211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:53.527441 28211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:53.528453 28211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "dcde863ea5d64dad841a6e5d5dbda1d5"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "0b178c9b60bee0422317ffd61eb46564"
server_key_iv: "99922701a43111e6a2a40d522877a867"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.528811 28211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "dcde863ea5d64dad841a6e5d5dbda1d5"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "0b178c9b60bee0422317ffd61eb46564"
server_key_iv: "99922701a43111e6a2a40d522877a867"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.530216 28211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:53.530985 28226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.531158 28211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:53.531261 28211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "dcde863ea5d64dad841a6e5d5dbda1d5"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "0b178c9b60bee0422317ffd61eb46564"
server_key_iv: "99922701a43111e6a2a40d522877a867"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.531329 28211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:53.547183 28211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:53.548849 28211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:53.548995 28211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:53.549196 28211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:53.549482 28211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:53.549535 28211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.549574 28211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:53.549602 28211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.555052 28211 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:34603
I20260927 11:08:53.555161 28339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:34603 every 8 connection(s)
I20260927 11:08:53.555449 28211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:08:53.556752 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 28211
I20260927 11:08:53.556828 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:08:53.557041 17791 external_mini_cluster.cc:1428] Setting key 213da6b14a94ca68093dd5fc349e4f4e
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
I20260927 11:08:53.565332 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.556111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:35703 (local address 127.17.95.254:37657)
0927 11:08:53.556191 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.556193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.556430 (+ 237us) server_negotiation.cc:407] Connection header received
0927 11:08:53.556999 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.557000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.557012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.557056 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:53.558229 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.558881 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.559946 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.560155 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.560229 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.562377 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:53.562378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:53.562379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:53.562386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:53.563901 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.564303 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.564304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.564304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.564359 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.564631 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.564632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.564633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.564774 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:53.564848 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.565154 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.565230 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:08:53.565502 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.556104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:37657 (local address 127.17.95.194:35703)
0927 11:08:53.556293 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.556347 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.556871 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.557119 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.557125 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.557964 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:53.558140 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.558145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.558972 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.558973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.559841 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.559843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.560234 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.560235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.560370 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.560879 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:53.560887 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:53.562284 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:53.563997 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:53.564003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:53.564005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:53.564216 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:53.564448 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:53.564449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:53.564449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:53.564556 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:53.564840 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:53.564845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:53.565004 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.565127 (+ 123us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.565273 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
I20260927 11:08:53.566045 28340 heartbeater.cc:347] Connected to a master server at 127.17.95.254:37657
I20260927 11:08:53.566120 28340 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:53.566300 28340 heartbeater.cc:511] Master 127.17.95.254:37657 requested a full tablet report, sending...
I20260927 11:08:53.566712 28015 ts_manager.cc:194] Registered new tserver with Master: dcde863ea5d64dad841a6e5d5dbda1d5 (127.17.95.194:34603)
I20260927 11:08:53.567195 28015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:35703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:08:53.581670 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37657
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:08:53.658906 28345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:08:53.659137 28345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:08:53.659174 28345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:08:53.660305 28345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:08:53.660365 28345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:08:53.660408 28345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:08:53.661754 28345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:37657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:08:53.661986 28345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:08:53.662199 28345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:08:53.666805 28353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.666816 28350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:08:53.666839 28351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:08:53.667124 28345 server_base.cc:1034] running on GCE node
I20260927 11:08:53.667336 28345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:08:53.667579 28345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:08:53.668743 28345 hybrid_clock.cc:648] HybridClock initialized: now 1790507333668728 us; error 33 us; skew 500 ppm
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:08:53.671195 28345 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:08:53.671607 28345 webserver.cc:460] Webserver started at http://127.17.95.195:33281/ using document root <none> and password file <none>
I20260927 11:08:53.671819 28345 fs_manager.cc:359] Metadata directory not provided
I20260927 11:08:53.671881 28345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:08:53.672006 28345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:08:53.672936 28345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "623cef0320aa4cbbb10b88add44c084d"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "fcc4741d1816db4b7fd95e523eea6845"
server_key_iv: "42d6b243753dbb02bfeaa2930e12f383"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.673278 28345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "623cef0320aa4cbbb10b88add44c084d"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "fcc4741d1816db4b7fd95e523eea6845"
server_key_iv: "42d6b243753dbb02bfeaa2930e12f383"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.674467 28345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:08:53.675158 28360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.675319 28345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:08:53.675402 28345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "623cef0320aa4cbbb10b88add44c084d"
format_stamp: "Formatted at 2026-09-27 11:08:53 on dist-test-slave-6j0v"
server_key: "fcc4741d1816db4b7fd95e523eea6845"
server_key_iv: "42d6b243753dbb02bfeaa2930e12f383"
server_key_version: "encryptionkey@0"
I20260927 11:08:53.675484 28345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:08:53.686051 28345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:08:53.688123 28345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:08:53.688269 28345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:08:53.688460 28345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:08:53.688740 28345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:08:53.688812 28345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.688853 28345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:08:53.688881 28345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:08:53.694319 28345 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:39329
I20260927 11:08:53.694370 28473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:39329 every 8 connection(s)
I20260927 11:08:53.694689 28345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:08:53.695670 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 28345
I20260927 11:08:53.695770 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:08:53.695962 17791 external_mini_cluster.cc:1428] Setting key d6ee5e37323cf16155f3747814c0426f
Sep 27 11:08:53 dist-test-slave-6j0v krb5kdc[27970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:08:53.703666 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.695410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:59201 (local address 127.17.95.254:37657)
0927 11:08:53.695477 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.695478 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.695658 (+ 180us) server_negotiation.cc:407] Connection header received
0927 11:08:53.696290 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.696291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.696304 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.696357 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:53.697380 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.698007 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.698920 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.699178 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.699229 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.701164 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:53.701165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:53.701165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:53.701171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:53.702576 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.702891 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.702891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.702892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.702951 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.703216 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.703217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.703217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.703314 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:53.703364 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.703553 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.703607 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:08:53.703881 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.695369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:37657 (local address 127.17.95.195:59201)
0927 11:08:53.695533 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.695572 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.696164 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.696433 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.696437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.697110 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:08:53.697309 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.697314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.698085 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.698086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.698852 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.698854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.699238 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.699239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.699334 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.699734 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:08:53.699741 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:08:53.701086 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:08:53.702639 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:53.702643 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:53.702645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:53.702831 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:53.703020 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:08:53.703020 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:08:53.703020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:08:53.703160 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:08:53.703366 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:08:53.703370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:08:53.703467 (+ 97us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.703541 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.703643 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260927 11:08:53.704360 28474 heartbeater.cc:347] Connected to a master server at 127.17.95.254:37657
I20260927 11:08:53.704452 28474 heartbeater.cc:464] Registering TS with master...
I20260927 11:08:53.704607 28474 heartbeater.cc:511] Master 127.17.95.254:37657 requested a full tablet report, sending...
I20260927 11:08:53.704959 28015 ts_manager.cc:194] Registered new tserver with Master: 623cef0320aa4cbbb10b88add44c084d (127.17.95.195:39329)
I20260927 11:08:53.705276 28015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:59201
I20260927 11:08:53.707562 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:08:53.714478 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.708860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.17.95.254:37657)
0927 11:08:53.708911 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.708913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.709017 (+ 104us) server_negotiation.cc:407] Connection header received
0927 11:08:53.709090 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.709090 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.709101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.709149 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:08:53.709680 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.710127 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.710887 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.711128 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.711198 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.712248 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:08:53.712250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:08:53.712250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:08:53.712257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:08:53.713380 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.713737 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.713738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.713738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.713787 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:08:53.713979 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:08:53.713980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:08:53.713981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:08:53.714113 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:08:53.714222 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.714320 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.714388 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 11:08:53.715775 28015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:08:53.717257 28015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:08:53.723209 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.719442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.17.95.195:39329)
0927 11:08:53.719795 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.719798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.719808 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:08:53.719842 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.719844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.719858 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.719970 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:53.720131 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.720835 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.722056 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.722780 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.722857 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.722917 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.723095 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.723133 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260927 11:08:53.723201 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.719380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:39329 (local address 127.0.0.1:53178)
0927 11:08:53.719646 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.719687 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.719751 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.719986 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.719989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.719991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:53.720049 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.720051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.720933 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.720934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.721865 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.721868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.722858 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.722859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.722938 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.722951 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.723057 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.723131 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":52,"threads_started":1}
I20260927 11:08:53.723621 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.719418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34603 (local address 127.0.0.1:51058)
0927 11:08:53.719846 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.719854 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.719899 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.720242 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.720242 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.720244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:53.720272 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.720274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.721515 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.721516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.722406 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.722408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.723378 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.723379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.723450 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.723455 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.723502 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.723541 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":27,"thread_start_us":87,"threads_started":1}
I20260927 11:08:53.723743 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.719420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:39203 (local address 127.0.0.1:34126)
0927 11:08:53.719889 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.719899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.719956 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.720219 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.720220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.720222 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:53.720272 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.720276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.721505 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.721506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.722583 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.722586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.723491 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.723492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.723552 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.723556 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.723615 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.723661 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":25,"thread_start_us":57,"threads_started":1}
I20260927 11:08:53.723770 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.719639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.17.95.193:39203)
0927 11:08:53.719789 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.719794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.720012 (+ 218us) server_negotiation.cc:407] Connection header received
0927 11:08:53.720053 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.720055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.720071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.720140 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:53.720348 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.721398 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.722667 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.723411 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.723479 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.723529 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.723627 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.723671 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260927 11:08:53.723904 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.719641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.17.95.194:34603)
0927 11:08:53.719843 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.719847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.719968 (+ 121us) server_negotiation.cc:407] Connection header received
0927 11:08:53.720070 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.720073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.720089 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.720155 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:53.720348 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.721417 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.722491 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.723229 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.723297 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.723344 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.723784 (+ 440us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.723832 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
I20260927 11:08:53.724236 28408 tablet_service.cc:1467] Processing CreateTablet for tablet 5097a4576402432ab29e384b2d760884 (DEFAULT_TABLE table=test-table [id=dc708e744974434dbb0289de4be52651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:53.724592 28408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5097a4576402432ab29e384b2d760884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:53.724535 28274 tablet_service.cc:1467] Processing CreateTablet for tablet 5097a4576402432ab29e384b2d760884 (DEFAULT_TABLE table=test-table [id=dc708e744974434dbb0289de4be52651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:53.724824 28274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5097a4576402432ab29e384b2d760884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:53.726967 28488 tablet_bootstrap.cc:492] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Bootstrap starting.
I20260927 11:08:53.727520 28140 tablet_service.cc:1467] Processing CreateTablet for tablet 5097a4576402432ab29e384b2d760884 (DEFAULT_TABLE table=test-table [id=dc708e744974434dbb0289de4be52651]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:08:53.727757 28488 tablet_bootstrap.cc:654] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:53.727795 28140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5097a4576402432ab29e384b2d760884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:08:53.728224 28488 log.cc:824] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:53.728868 28488 tablet_bootstrap.cc:492] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: No bootstrap required, opened a new log
I20260927 11:08:53.728919 28488 ts_tablet_manager.cc:1397] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:08:53.730221 28491 tablet_bootstrap.cc:492] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: Bootstrap starting.
I20260927 11:08:53.730341 28489 tablet_bootstrap.cc:492] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Bootstrap starting.
I20260927 11:08:53.730440 28488 raft_consensus.cc:348] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.730690 28488 raft_consensus.cc:374] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:53.730728 28488 raft_consensus.cc:729] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 623cef0320aa4cbbb10b88add44c084d, State: Initialized, Role: FOLLOWER
I20260927 11:08:53.730872 28488 consensus_queue.cc:260] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.730952 28491 tablet_bootstrap.cc:654] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:53.731038 28489 tablet_bootstrap.cc:654] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Neither blocks nor log segments found. Creating new log.
I20260927 11:08:53.731184 28488 ts_tablet_manager.cc:1428] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:53.731312 28474 heartbeater.cc:503] Master 127.17.95.254:37657 was elected leader, sending a full tablet report...
I20260927 11:08:53.731575 28491 log.cc:824] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:53.731966 28489 log.cc:824] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Log is configured to *not* fsync() on all Append() calls
I20260927 11:08:53.732277 28491 tablet_bootstrap.cc:492] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: No bootstrap required, opened a new log
I20260927 11:08:53.732358 28491 ts_tablet_manager.cc:1397] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:08:53.732635 28489 tablet_bootstrap.cc:492] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: No bootstrap required, opened a new log
I20260927 11:08:53.732686 28489 ts_tablet_manager.cc:1397] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:53.733883 28491 raft_consensus.cc:348] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.734035 28491 raft_consensus.cc:374] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:53.734079 28491 raft_consensus.cc:729] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70c9f1050d214ddfad97d144c1bf16da, State: Initialized, Role: FOLLOWER
I20260927 11:08:53.734192 28491 consensus_queue.cc:260] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.734133 28489 raft_consensus.cc:348] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.734280 28489 raft_consensus.cc:374] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:08:53.734332 28489 raft_consensus.cc:729] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcde863ea5d64dad841a6e5d5dbda1d5, State: Initialized, Role: FOLLOWER
I20260927 11:08:53.734462 28491 ts_tablet_manager.cc:1428] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:08:53.734443 28489 consensus_queue.cc:260] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.734568 28206 heartbeater.cc:503] Master 127.17.95.254:37657 was elected leader, sending a full tablet report...
I20260927 11:08:53.734724 28489 ts_tablet_manager.cc:1428] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:08:53.734768 28340 heartbeater.cc:503] Master 127.17.95.254:37657 was elected leader, sending a full tablet report...
W20260927 11:08:53.806111 28341 tablet.cc:2369] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:08:53.898324 28207 tablet.cc:2369] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:08:53.945204 28475 tablet.cc:2369] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:08:53.953298 28496 raft_consensus.cc:482] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:08:53.953409 28496 raft_consensus.cc:504] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.953855 28496 leader_election.cc:288] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 623cef0320aa4cbbb10b88add44c084d (127.17.95.195:39329), 70c9f1050d214ddfad97d144c1bf16da (127.17.95.193:39203)
I20260927 11:08:53.957894 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.954117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:41729 (local address 127.17.95.193:39203)
0927 11:08:53.954205 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.954207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.954316 (+ 109us) server_negotiation.cc:407] Connection header received
0927 11:08:53.954363 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.954364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.954375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.954431 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:53.954574 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.955342 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.956921 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.957535 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.957616 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.957625 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.957787 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.957826 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:08:53.957894 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.954139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:57657 (local address 127.17.95.195:39329)
0927 11:08:53.954181 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:53.954183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:53.954212 (+ 29us) server_negotiation.cc:407] Connection header received
0927 11:08:53.954240 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:53.954241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:53.954252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:53.954346 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:08:53.954569 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.955342 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.956922 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.957535 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.957617 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.957625 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:53.957786 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 11:08:53.957827 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:08:53.957911 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.954032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:39329 (local address 127.17.95.194:57657)
0927 11:08:53.954091 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.954103 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.954148 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.954369 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.954369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.954373 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:53.954468 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.954470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.955446 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.955447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.956781 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.956784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.957627 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.957627 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.957715 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.957718 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.957764 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.957826 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":113}
I20260927 11:08:53.957917 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:53.954035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:39203 (local address 127.17.95.194:41729)
0927 11:08:53.954242 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:53.954256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 11:08:53.954298 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:08:53.954445 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:08:53.954446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:08:53.954448 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:08:53.954501 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.954502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.955446 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.955447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.956781 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:08:53.956784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:08:53.957627 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:08:53.957627 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:08:53.957704 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:53.957708 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:08:53.957764 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:08:53.957821 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":99,"thread_start_us":102,"threads_started":1}
I20260927 11:08:53.958292 28428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5097a4576402432ab29e384b2d760884" candidate_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "623cef0320aa4cbbb10b88add44c084d" is_pre_election: true
I20260927 11:08:53.958320 28160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5097a4576402432ab29e384b2d760884" candidate_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70c9f1050d214ddfad97d144c1bf16da" is_pre_election: true
I20260927 11:08:53.958456 28428 raft_consensus.cc:2454] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcde863ea5d64dad841a6e5d5dbda1d5 in term 0.
I20260927 11:08:53.958472 28160 raft_consensus.cc:2454] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcde863ea5d64dad841a6e5d5dbda1d5 in term 0.
I20260927 11:08:53.958715 28230 leader_election.cc:302] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 623cef0320aa4cbbb10b88add44c084d, dcde863ea5d64dad841a6e5d5dbda1d5; no voters:
I20260927 11:08:53.958865 28496 raft_consensus.cc:2781] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:08:53.958921 28496 raft_consensus.cc:482] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:08:53.958949 28496 raft_consensus.cc:3037] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:53.959741 28496 raft_consensus.cc:504] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.959952 28496 leader_election.cc:288] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [CANDIDATE]: Term 1 election: Requested vote from peers 623cef0320aa4cbbb10b88add44c084d (127.17.95.195:39329), 70c9f1050d214ddfad97d144c1bf16da (127.17.95.193:39203)
I20260927 11:08:53.960041 28428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5097a4576402432ab29e384b2d760884" candidate_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "623cef0320aa4cbbb10b88add44c084d"
I20260927 11:08:53.960116 28428 raft_consensus.cc:3037] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:53.960103 28160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5097a4576402432ab29e384b2d760884" candidate_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70c9f1050d214ddfad97d144c1bf16da"
I20260927 11:08:53.960191 28160 raft_consensus.cc:3037] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:08:53.960785 28428 raft_consensus.cc:2454] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcde863ea5d64dad841a6e5d5dbda1d5 in term 1.
I20260927 11:08:53.960953 28230 leader_election.cc:302] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 623cef0320aa4cbbb10b88add44c084d, dcde863ea5d64dad841a6e5d5dbda1d5; no voters:
I20260927 11:08:53.961045 28496 raft_consensus.cc:2781] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:08:53.961042 28160 raft_consensus.cc:2454] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcde863ea5d64dad841a6e5d5dbda1d5 in term 1.
I20260927 11:08:53.961197 28496 raft_consensus.cc:686] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 1 LEADER]: Becoming Leader. State: Replica: dcde863ea5d64dad841a6e5d5dbda1d5, State: Running, Role: LEADER
I20260927 11:08:53.961295 28496 consensus_queue.cc:237] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } }
I20260927 11:08:53.962030 28015 catalog_manager.cc:5495] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcde863ea5d64dad841a6e5d5dbda1d5 (127.17.95.194). New cstate: current_term: 1 leader_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcde863ea5d64dad841a6e5d5dbda1d5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 } health_report { overall_health: UNKNOWN } } }
I20260927 11:08:54.004184 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:54.002012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.17.95.194:34603)
0927 11:08:54.002072 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:08:54.002074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:08:54.002081 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:08:54.002112 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:08:54.002113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:08:54.002127 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:08:54.002197 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:08:54.002309 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:54.002772 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:54.003504 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:08:54.003713 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:08:54.003779 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:08:54.003902 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:08:54.004001 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:08:54.004059 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 11:08:54.004094 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 11:08:54.004112 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:08:54.006062 28428 raft_consensus.cc:1261] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 1 FOLLOWER]: Refusing update from remote peer dcde863ea5d64dad841a6e5d5dbda1d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:54.006047 28160 raft_consensus.cc:1261] T 5097a4576402432ab29e384b2d760884 P 70c9f1050d214ddfad97d144c1bf16da [term 1 FOLLOWER]: Refusing update from remote peer dcde863ea5d64dad841a6e5d5dbda1d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:08:54.006377 28496 consensus_queue.cc:1035] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70c9f1050d214ddfad97d144c1bf16da" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:54.006510 28498 consensus_queue.cc:1035] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "623cef0320aa4cbbb10b88add44c084d" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 39329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:08:54.009429 28503 mvcc.cc:205] Tried to move back new op lower bound from 7333918040085872640 to 7333918039905722368. Current Snapshot: MvccSnapshot[applied={T|T < 7333918040085872640}]
I20260927 11:08:54.009485 28502 mvcc.cc:205] Tried to move back new op lower bound from 7333918040085872640 to 7333918039905722368. Current Snapshot: MvccSnapshot[applied={T|T < 7333918040085872640}]
I20260927 11:08:54.010972 28014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:54.011091 28014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:08:54.011878 28014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 333d7a6b925b4a698c52cf98df274276: Sending DeleteTablet for 3 replicas of tablet 5097a4576402432ab29e384b2d760884
I20260927 11:08:54.012204 28408 tablet_service.cc:1514] Processing DeleteTablet for tablet 5097a4576402432ab29e384b2d760884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:54 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:53178
I20260927 11:08:54.012231 28140 tablet_service.cc:1514] Processing DeleteTablet for tablet 5097a4576402432ab29e384b2d760884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:54 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34126
I20260927 11:08:54.012224 28274 tablet_service.cc:1514] Processing DeleteTablet for tablet 5097a4576402432ab29e384b2d760884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:08:54 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:51058
I20260927 11:08:54.012424 28512 tablet_replica.cc:330] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: stopping tablet replica
I20260927 11:08:54.012521 28512 raft_consensus.cc:2229] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:08:54.012646 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 28077
I20260927 11:08:54.012739 28510 tablet_replica.cc:330] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: stopping tablet replica
I20260927 11:08:54.012826 28510 raft_consensus.cc:2229] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:08:54.012912 28510 raft_consensus.cc:2258] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:54.013377 28512 raft_consensus.cc:2258] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:08:54.013833 28510 ts_tablet_manager.cc:1905] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:54.015702 28510 ts_tablet_manager.cc:1918] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:54.015774 28510 log.cc:1196] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/5097a4576402432ab29e384b2d760884
I20260927 11:08:54.016060 28510 ts_tablet_manager.cc:1939] T 5097a4576402432ab29e384b2d760884 P 623cef0320aa4cbbb10b88add44c084d: Deleting consensus metadata
I20260927 11:08:54.016299 28512 ts_tablet_manager.cc:1905] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:08:54.017553 28512 ts_tablet_manager.cc:1918] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:08:54.017601 28512 log.cc:1196] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/5097a4576402432ab29e384b2d760884
I20260927 11:08:54.017786 28512 ts_tablet_manager.cc:1939] T 5097a4576402432ab29e384b2d760884 P dcde863ea5d64dad841a6e5d5dbda1d5: Deleting consensus metadata
I20260927 11:08:54.018106 28001 catalog_manager.cc:4841] TS dcde863ea5d64dad841a6e5d5dbda1d5 (127.17.95.194:34603): tablet 5097a4576402432ab29e384b2d760884 (table test-table [id=dc708e744974434dbb0289de4be52651]) successfully deleted
I20260927 11:08:54.019214 28002 catalog_manager.cc:4841] TS 623cef0320aa4cbbb10b88add44c084d (127.17.95.195:39329): tablet 5097a4576402432ab29e384b2d760884 (table test-table [id=dc708e744974434dbb0289de4be52651]) successfully deleted
W20260927 11:08:54.020020 27999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:39203 (error 108)
I20260927 11:08:54.020272 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 28211
I20260927 11:08:54.023439 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:08:54.023298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:39203 (local address 127.0.0.1:34132)
0927 11:08:54.023354 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:08:54.023393 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:39203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 11:08:54.023588 27999 catalog_manager.cc:4584] TS 70c9f1050d214ddfad97d144c1bf16da (127.17.95.193:39203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5097a4576402432ab29e384b2d760884: Network error: Client connection negotiation failed: client connection to 127.17.95.193:39203: connect: Connection refused (error 111)
I20260927 11:08:54.026583 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 28345
I20260927 11:08:54.030900 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 27983
2026-09-27T11:08:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2053 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:08:54 dist-test-slave-6j0v krb5kdc[28516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:08:54 dist-test-slave-6j0v krb5kdc[28516](info): set up 2 sockets
Sep 27 11:08:54 dist-test-slave-6j0v krb5kdc[28516](info): commencing operation
krb5kdc: starting...
W20260927 11:08:55.071961 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:08:55 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:08:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/rangeradmin_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/rangeradmin_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/rangerlookup_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/rangerlookup_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
I20260927 11:08:55.221474 17791 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWCkVYM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/postgres -l logfile start
2026-09-27 11:08:57.572 UTC [28546] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 11:08:57.572 UTC [28546] LOG: listening on IPv4 address "127.17.95.212", port 59423
2026-09-27 11:08:57.575 UTC [28546] LOG: listening on Unix socket "/tmp/.s.PGSQL.59423"
2026-09-27 11:08:57.609 UTC [28549] LOG: database system was shut down at 2026-09-27 11:08:56 UTC
2026-09-27 11:08:57.613 UTC [28546] LOG: database system is ready to accept connections
I20260927 11:08:58.543951 17791 mini_postgres.cc:96] Postgres bound to 59423
2026-09-27 11:08:58.550 UTC [28557] FATAL: database "slave" does not exist
127.17.95.212:59423 - accepting connections
I20260927 11:08:58.550509 17791 mini_ranger.cc:161] Starting Ranger...
I20260927 11:08:58.564594 17791 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 11:08:58.996367 17791 mini_ranger.cc:87] Created ranger Postgres database
I20260927 11:08:58.996459 17791 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-admin
2026-09-27 11:08:59,273 [I] DB FLAVOR :POSTGRES
2026-09-27 11:08:59,274 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 11:08:59,274 [I] Checking connection..
2026-09-27 11:08:59,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 11:08:59,514 [I] Checking connection passed.
2026-09-27 11:08:59,515 [I] --------- Verifying version history table ---------
2026-09-27 11:08:59,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 11:08:59,770 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 11:08:59,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 11:09:00,008 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 11:09:00,009 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 11:09:00,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 11:09:00.237 UTC [28621] WARNING: there is no transaction in progress
2026-09-27 11:09:00,243 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 11:09:00,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 11:09:00,486 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 11:09:00,487 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 11:09:00,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 11:09:00,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 11:09:00,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6j0v.c.gcp-upstream.internal','N') ;"
2026-09-27 11:09:01,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 11:09:01,477 [I] Table x_portal_user does not exist in database ranger
2026-09-27 11:09:01,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 11:09:01,722 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 11:09:01,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 11:09:01,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 11:09:02,169 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 11:09:02,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 11:09:03.089 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.105 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.120 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.134 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.149 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.165 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.215 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.221 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.234 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.244 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.254 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.268 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.277 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.283 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.289 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03.294 UTC [28729] WARNING: there is no transaction in progress
2026-09-27 11:09:03,875 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 11:09:03,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 11:09:04,117 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 11:09:04,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 11:09:04,365 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 11:09:04,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 11:09:04,614 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 11:09:04,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 11:09:04,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 11:09:05,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6j0v.c.gcp-upstream.internal';"
2026-09-27 11:09:05,316 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 11:09:05,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 11:09:05,563 [I] DB_PATCHES have already been applied
I20260927 11:09:05.568914 17791 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWCkVYM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 11:09:05.568974 17791 mini_ranger.cc:193] Using host: 127.17.95.212
I20260927 11:09:05.572168 17791 mini_ranger.cc:227] Ranger admin URL: http://127.17.95.212:40371
Sep 27, 2026 11:09:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 11:09:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 11:09:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 11:09:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 11:09:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 11:09:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 11:09:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 11:09:06.704881 17791 mini_ranger.cc:160] Time spent starting Ranger: real 8.154s user 0.001s sys 0.007s
Sep 27, 2026 11:09:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 11:09:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 27, 2026 11:09:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 11:09:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 11:09:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 11:09:26.752564 17791 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/rangerkms_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/rangerkms_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.95.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.95.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 11:09:26 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507366, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 11:09:26.832656 17791 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 11:09:26.841334 17791 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 11:09:27.628711 17791 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 11:09:27.628793 17791 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms
I20260927 11:09:27.628810 17791 mini_ranger_kms.cc:227] Using postgres at 127.17.95.212:59423
2026-09-27 11:09:27,802 [I] DB FLAVOR :POSTGRES
2026-09-27 11:09:27,802 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 11:09:27,803 [I] Checking connection
2026-09-27 11:09:27,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 11:09:28,053 [I] connection success
2026-09-27 11:09:28,053 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 11:09:28,053 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 11:09:28,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 11:09:28,278 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 11:09:28,278 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 11:09:28,278 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 11:09:28,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:59423/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 11:09:28,541 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 11:09:28.963411 17791 mini_ranger_kms.cc:310] Created kms service
I20260927 11:09:29.150388 17791 mini_ranger_kms.cc:326] Created kudu user
I20260927 11:09:29.191363 17791 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 11:09:29 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507366, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:09:29.592406 17791 mini_ranger_kms.cc:384] Added ranger policy
I20260927 11:09:29.592530 17791 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 11:09:29.592554 17791 mini_ranger_kms.cc:240] Using host: 127.17.95.212
I20260927 11:09:29.596036 17791 mini_ranger_kms.cc:276] Ranger KMS PID: 28909
I20260927 11:09:29.596159 17791 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.95.212:48009
Sep 27, 2026 11:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 11:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 11:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 11:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 11:09:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 11:09:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 27 11:09:31 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790507371, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.95.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:09:31.809713 17791 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.977s user 0.014s sys 0.142s
I20260927 11:09:31.809829 17791 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 11:09:31.809906 17791 mini_ranger_kms.cc:401] 127.17.95.212:48009/kms/v1/keys
Sep 27 11:09:32 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507371, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.95.212@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
Sep 27 11:09:32 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507371, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.95.212@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
Sep 27 11:09:34 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:09:34.733459 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35301
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:35301
--ranger_config_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48009/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:09:34.813616 28960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:09:34.813987 28960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:09:34.814064 28960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:09:34.815811 28960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:09:34.815882 28960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:09:34.815907 28960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:09:34.815945 28960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:09:34.815981 28960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:09:34.818476 28960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.95.212:48009/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:35301
--ranger_config_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:09:34.818732 28960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:09:34.818964 28960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:09:34.825634 28965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:09:34.825696 28968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:09:34.825760 28966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:09:34.826187 28960 server_base.cc:1034] running on GCE node
I20260927 11:09:34.826421 28960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:09:34.826728 28960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:09:34.827875 28960 hybrid_clock.cc:648] HybridClock initialized: now 1790507374827864 us; error 28 us; skew 500 ppm
Sep 27 11:09:34 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507374, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:09:34.833374 28960 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:09:34.833874 28960 webserver.cc:460] Webserver started at http://127.17.95.254:33111/ using document root <none> and password file <none>
I20260927 11:09:34.834111 28960 fs_manager.cc:359] Metadata directory not provided
I20260927 11:09:34.834151 28960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:09:34.834280 28960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:09:34 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507374, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:09:34.885640 28960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "375815d0a5af4034931af1bc5a7ad74b"
format_stamp: "Formatted at 2026-09-27 11:09:34 on dist-test-slave-6j0v"
server_key: "ad6179a6b992258e9215508e6de3dff6"
server_key_iv: "e82ab77972c17476b7765b094e0ba0e8"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:34.886053 28960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "375815d0a5af4034931af1bc5a7ad74b"
format_stamp: "Formatted at 2026-09-27 11:09:34 on dist-test-slave-6j0v"
server_key: "ad6179a6b992258e9215508e6de3dff6"
server_key_iv: "e82ab77972c17476b7765b094e0ba0e8"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:34.887662 28960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260927 11:09:34.901844 28975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:34.902055 28960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:09:34.902160 28960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "375815d0a5af4034931af1bc5a7ad74b"
format_stamp: "Formatted at 2026-09-27 11:09:34 on dist-test-slave-6j0v"
server_key: "ad6179a6b992258e9215508e6de3dff6"
server_key_iv: "e82ab77972c17476b7765b094e0ba0e8"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:34.902246 28960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:09:34.911684 28960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:09:34.913504 28960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:09:34.913648 28960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:09:34.918253 28960 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:35301
I20260927 11:09:34.918298 29027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:35301 every 8 connection(s)
I20260927 11:09:34.918643 28960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:09:34.919461 29028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:09:34.922432 29028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b: Bootstrap starting.
I20260927 11:09:34.923144 29028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b: Neither blocks nor log segments found. Creating new log.
I20260927 11:09:34.923440 29028 log.cc:824] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b: Log is configured to *not* fsync() on all Append() calls
I20260927 11:09:34.924199 29028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b: No bootstrap required, opened a new log
I20260927 11:09:34.926980 29028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375815d0a5af4034931af1bc5a7ad74b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35301 } }
I20260927 11:09:34.927172 29028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:09:34.927206 29028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375815d0a5af4034931af1bc5a7ad74b, State: Initialized, Role: FOLLOWER
I20260927 11:09:34.927371 29028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375815d0a5af4034931af1bc5a7ad74b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35301 } }
I20260927 11:09:34.927441 29028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:09:34.927469 29028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:09:34.927520 29028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:09:34.928131 29028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375815d0a5af4034931af1bc5a7ad74b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35301 } }
I20260927 11:09:34.928231 29028 leader_election.cc:302] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375815d0a5af4034931af1bc5a7ad74b; no voters:
I20260927 11:09:34.928385 29028 leader_election.cc:288] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:09:34.928447 29031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:09:34.928615 29028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:09:34.928674 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 28960
I20260927 11:09:34.928680 29031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [term 1 LEADER]: Becoming Leader. State: Replica: 375815d0a5af4034931af1bc5a7ad74b, State: Running, Role: LEADER
I20260927 11:09:34.928776 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:09:34.928835 29031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375815d0a5af4034931af1bc5a7ad74b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35301 } }
I20260927 11:09:34.928975 17791 external_mini_cluster.cc:1428] Setting key 874b538c93b80fa4b83f7aa447c9f5dc
I20260927 11:09:34.929554 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375815d0a5af4034931af1bc5a7ad74b. Latest consensus state: current_term: 1 leader_uuid: "375815d0a5af4034931af1bc5a7ad74b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375815d0a5af4034931af1bc5a7ad74b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35301 } } }
I20260927 11:09:34.929685 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [sys.catalog]: This master's current role is: LEADER
I20260927 11:09:34.929750 29033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "375815d0a5af4034931af1bc5a7ad74b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375815d0a5af4034931af1bc5a7ad74b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35301 } } }
I20260927 11:09:34.929859 29033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b [sys.catalog]: This master's current role is: LEADER
I20260927 11:09:34.929919 29042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:09:34.930426 29042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:09:34.931331 29028 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 11:09:34.933391 29042 catalog_manager.cc:1347] Generated new cluster ID: 573f9747bcd84053b72d8519eb685f03
I20260927 11:09:34.933432 29042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:09:34 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:09:34.944977 29042 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:09:34.945894 29042 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:09:34.947022 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:34.929435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.17.95.254:35301)
0927 11:09:34.930030 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:34.930039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:34.930071 (+ 32us) server_negotiation.cc:407] Connection header received
0927 11:09:34.930185 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:34.930193 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:34.930254 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:34.931215 (+ 961us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:09:34.931310 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:34.932680 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:34.935194 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:34.935783 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:34.935889 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:34.939913 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:09:34.939968 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:09:34.939969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:09:34.940053 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:09:34.944578 (+ 4525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:34.945016 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:34.945022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:34.945023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:34.945151 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:34.945411 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:34.945411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:34.945412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:34.945757 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:09:34.946312 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:34.946399 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 11:09:34.946720 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":98,"threads_started":1}
I20260927 11:09:34.951849 29042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b: Generated new TSK 0
I20260927 11:09:34.952040 29042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:09:35.754485 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:35301
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:09:35.862192 29073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:09:35.862531 29073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:09:35.862609 29073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:09:35.863930 29073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:09:35.863997 29073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:09:35.864046 29073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:09:35.865408 29073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.95.212:48009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:35301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:09:35.865643 29073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:09:35.865890 29073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:09:35.871820 29079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:09:35.872053 29081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:09:35.872377 29073 server_base.cc:1034] running on GCE node
W20260927 11:09:35.872432 29078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:09:35.872677 29073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:09:35.872912 29073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:09:35.874091 29073 hybrid_clock.cc:648] HybridClock initialized: now 1790507375874078 us; error 31 us; skew 500 ppm
Sep 27 11:09:35 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507375, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:09:35.876801 29073 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:09:35.877285 29073 webserver.cc:460] Webserver started at http://127.17.95.193:35531/ using document root <none> and password file <none>
I20260927 11:09:35.877514 29073 fs_manager.cc:359] Metadata directory not provided
I20260927 11:09:35.877584 29073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:09:35.877717 29073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:09:35 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507375, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:09:35.889158 29073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "8e7e0167e1cc442596974b3b4e83cd0a"
format_stamp: "Formatted at 2026-09-27 11:09:35 on dist-test-slave-6j0v"
server_key: "8e9dd8967419f347a03ec16a3041e48a"
server_key_iv: "180d1bd9821e1a50e5e0f4c676a50c5e"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:35.889636 29073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "8e7e0167e1cc442596974b3b4e83cd0a"
format_stamp: "Formatted at 2026-09-27 11:09:35 on dist-test-slave-6j0v"
server_key: "8e9dd8967419f347a03ec16a3041e48a"
server_key_iv: "180d1bd9821e1a50e5e0f4c676a50c5e"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:35.891409 29073 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260927 11:09:35.907182 29088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:35.907414 29073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260927 11:09:35.907508 29073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "8e7e0167e1cc442596974b3b4e83cd0a"
format_stamp: "Formatted at 2026-09-27 11:09:35 on dist-test-slave-6j0v"
server_key: "8e9dd8967419f347a03ec16a3041e48a"
server_key_iv: "180d1bd9821e1a50e5e0f4c676a50c5e"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:35.907582 29073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:09:35.930339 29073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:09:35.932715 29073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:09:35.932910 29073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:09:35.933226 29073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:09:35.933632 29073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:09:35.933719 29073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:35.933809 29073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:09:35.933877 29073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:35.940972 29073 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:39185
I20260927 11:09:35.941439 29073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:09:35.943881 29203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:39185 every 8 connection(s)
I20260927 11:09:35.945927 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 29073
I20260927 11:09:35.946038 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:09:35.946214 17791 external_mini_cluster.cc:1428] Setting key a4b7f2bc5e33d96d8a14eb401a6bcea0
Sep 27 11:09:35 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507375, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
I20260927 11:09:35.961822 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:35.944530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:35301 (local address 127.17.95.193:53605)
0927 11:09:35.944971 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:35.945021 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:35.945675 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:35.946092 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:35.946097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:35.946340 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:09:35.946617 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:35.946624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:35.947623 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:35.947624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:35.948834 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:35.948836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:35.950689 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:35.950690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:35.950848 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:35.951445 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:09:35.951455 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:09:35.953139 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:09:35.955050 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:09:35.955087 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:09:35.955089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:09:35.955371 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:09:35.956614 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:09:35.956615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:09:35.956616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:09:35.956741 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:09:35.961164 (+ 4423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:09:35.961171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:09:35.961329 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 11:09:35.961443 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 11:09:35.961593 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":106,"threads_started":1}
I20260927 11:09:35.962558 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:35.944775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:53605 (local address 127.17.95.254:35301)
0927 11:09:35.944940 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:35.944942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:35.945123 (+ 181us) server_negotiation.cc:407] Connection header received
0927 11:09:35.945879 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:35.945880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:35.945894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:35.945958 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:09:35.946727 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:35.947507 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:35.948927 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:35.949240 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:35.949309 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:35.953243 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:09:35.953244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:09:35.953244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:09:35.953251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:09:35.954968 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:35.956476 (+ 1508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:35.956477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:35.956477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:35.956552 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:35.960915 (+ 4363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:35.960916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:35.960917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:35.961070 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:09:35.962337 (+ 1267us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:35.962380 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 11:09:35.962462 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260927 11:09:35.963155 29204 heartbeater.cc:347] Connected to a master server at 127.17.95.254:35301
I20260927 11:09:35.963248 29204 heartbeater.cc:464] Registering TS with master...
I20260927 11:09:35.963428 29204 heartbeater.cc:511] Master 127.17.95.254:35301 requested a full tablet report, sending...
I20260927 11:09:35.963974 28992 ts_manager.cc:194] Registered new tserver with Master: 8e7e0167e1cc442596974b3b4e83cd0a (127.17.95.193:39185)
I20260927 11:09:35.964824 28992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:53605
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:09:35.982425 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:35301
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 11:09:36.111413 29210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:09:36.111866 29210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:09:36.111963 29210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:09:36.114179 29210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:09:36.114283 29210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:09:36.114362 29210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:09:36.116508 29210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.95.212:48009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:35301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:09:36.117030 29210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:09:36.117348 29210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:09:36.125903 29220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:09:36.125883 29217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:09:36.126114 29218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:09:36.126451 29210 server_base.cc:1034] running on GCE node
I20260927 11:09:36.126672 29210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:09:36.126924 29210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:09:36.128084 29210 hybrid_clock.cc:648] HybridClock initialized: now 1790507376128036 us; error 58 us; skew 500 ppm
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:09:36.131861 29210 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:09:36.132387 29210 webserver.cc:460] Webserver started at http://127.17.95.194:44589/ using document root <none> and password file <none>
I20260927 11:09:36.132627 29210 fs_manager.cc:359] Metadata directory not provided
I20260927 11:09:36.132704 29210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:09:36.132831 29210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:09:36.142987 29210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e"
format_stamp: "Formatted at 2026-09-27 11:09:36 on dist-test-slave-6j0v"
server_key: "d0060e1b028e49cd4dfd542865031a99"
server_key_iv: "6382136e0ece138c0860011cc4a2aa60"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:36.143468 29210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e"
format_stamp: "Formatted at 2026-09-27 11:09:36 on dist-test-slave-6j0v"
server_key: "d0060e1b028e49cd4dfd542865031a99"
server_key_iv: "6382136e0ece138c0860011cc4a2aa60"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:36.147465 29210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260927 11:09:36.159833 29227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:36.160020 29210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:09:36.160099 29210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e"
format_stamp: "Formatted at 2026-09-27 11:09:36 on dist-test-slave-6j0v"
server_key: "d0060e1b028e49cd4dfd542865031a99"
server_key_iv: "6382136e0ece138c0860011cc4a2aa60"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:36.160172 29210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:09:36.173349 29210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:09:36.175817 29210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:09:36.176010 29210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:09:36.176291 29210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:09:36.176704 29210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:09:36.176791 29210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:36.176862 29210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:09:36.176918 29210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:36.183948 29210 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:38435
I20260927 11:09:36.184451 29210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:09:36.184928 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:38435 every 8 connection(s)
I20260927 11:09:36.192579 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 29210
I20260927 11:09:36.192692 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:09:36.192888 17791 external_mini_cluster.cc:1428] Setting key fa2c243128a463e767d77e024f2930b3
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:09:36.196703 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.185562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:51499 (local address 127.17.95.254:35301)
0927 11:09:36.185616 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.185619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.185846 (+ 227us) server_negotiation.cc:407] Connection header received
0927 11:09:36.186635 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.186637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.186651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.186715 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:09:36.187349 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.187845 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.188829 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.189036 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.189115 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.193526 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:09:36.193527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:09:36.193528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:09:36.193535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:09:36.195209 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:36.195700 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:36.195701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:36.195702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:36.195754 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:36.195989 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:36.195990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:36.195991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:36.196110 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:09:36.196149 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.196494 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.196591 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:09:36.197373 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.185545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:35301 (local address 127.17.95.194:51499)
0927 11:09:36.185730 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.185771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.186533 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.186785 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.186790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.186972 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:09:36.187231 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.187236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.187935 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.187936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.188726 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.188728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.191161 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.191162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.191302 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.191823 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:09:36.191833 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:09:36.193406 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:09:36.195310 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:09:36.195316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:09:36.195318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:09:36.195598 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:09:36.195816 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:09:36.195816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:09:36.195817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:09:36.195925 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:09:36.196191 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:09:36.196197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:09:36.196332 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.197015 (+ 683us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.197165 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":119,"threads_started":1}
I20260927 11:09:36.197976 29341 heartbeater.cc:347] Connected to a master server at 127.17.95.254:35301
I20260927 11:09:36.198058 29341 heartbeater.cc:464] Registering TS with master...
I20260927 11:09:36.198217 29341 heartbeater.cc:511] Master 127.17.95.254:35301 requested a full tablet report, sending...
I20260927 11:09:36.198602 28992 ts_manager.cc:194] Registered new tserver with Master: 8d3d832e5c014f91a7ea69fa47cd3c1e (127.17.95.194:38435)
I20260927 11:09:36.199075 28992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:51499
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:09:36.230402 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:35301
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:09:36.345803 29347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:09:36.346123 29347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:09:36.346204 29347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:09:36.347445 29347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:09:36.347507 29347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:09:36.347622 29347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:09:36.349018 29347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.95.212:48009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:35301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:09:36.349264 29347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:09:36.349501 29347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:09:36.355705 29355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:09:36.355989 29347 server_base.cc:1034] running on GCE node
W20260927 11:09:36.360016 29352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:09:36.360030 29353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:09:36.360384 29347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:09:36.360646 29347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:09:36.363884 29347 hybrid_clock.cc:648] HybridClock initialized: now 1790507376363867 us; error 24 us; skew 500 ppm
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:09:36.366752 29347 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:09:36.367220 29347 webserver.cc:460] Webserver started at http://127.17.95.195:32947/ using document root <none> and password file <none>
I20260927 11:09:36.367470 29347 fs_manager.cc:359] Metadata directory not provided
I20260927 11:09:36.367543 29347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:09:36.367689 29347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:09:36.384711 29347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "30d76b0e4f4e475590d9407c693ece1b"
format_stamp: "Formatted at 2026-09-27 11:09:36 on dist-test-slave-6j0v"
server_key: "9613945546bfb6fb3052eec1fc6f6ae4"
server_key_iv: "b7fad97ef161cb0992f7d985c33793db"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:36.385257 29347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "30d76b0e4f4e475590d9407c693ece1b"
format_stamp: "Formatted at 2026-09-27 11:09:36 on dist-test-slave-6j0v"
server_key: "9613945546bfb6fb3052eec1fc6f6ae4"
server_key_iv: "b7fad97ef161cb0992f7d985c33793db"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:36.390997 29347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260927 11:09:36.406837 29362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:36.407137 29347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 11:09:36.407234 29347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "30d76b0e4f4e475590d9407c693ece1b"
format_stamp: "Formatted at 2026-09-27 11:09:36 on dist-test-slave-6j0v"
server_key: "9613945546bfb6fb3052eec1fc6f6ae4"
server_key_iv: "b7fad97ef161cb0992f7d985c33793db"
server_key_version: "kuduclusterkey@0"
I20260927 11:09:36.407339 29347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:09:36.426183 29347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:09:36.428323 29347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:09:36.428478 29347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:09:36.428730 29347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:09:36.429147 29347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:09:36.429203 29347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:36.429268 29347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:09:36.429297 29347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:09:36.436590 29347 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:33525
I20260927 11:09:36.436951 29347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:09:36.437474 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 29347
I20260927 11:09:36.437579 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:09:36.437798 17791 external_mini_cluster.cc:1428] Setting key bc39be7f6c959cd11a78c4ebd64540ce
I20260927 11:09:36.437829 29475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:33525 every 8 connection(s)
Sep 27 11:09:36 dist-test-slave-6j0v krb5kdc[28516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507376, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:09:36.447374 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.438630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:50399 (local address 127.17.95.254:35301)
0927 11:09:36.438670 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.438672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.438680 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:09:36.439362 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.439363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.439374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.439418 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:09:36.439956 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.440487 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.441510 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.441742 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.441825 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.444170 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:09:36.444172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:09:36.444172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:09:36.444179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:09:36.445823 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:36.446299 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:36.446301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:36.446301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:36.446357 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:36.446622 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:36.446623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:36.446623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:36.446748 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:09:36.446788 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.447179 (+ 391us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.447276 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:09:36.448113 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.438337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:35301 (local address 127.17.95.195:50399)
0927 11:09:36.438515 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.438564 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.439261 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.439487 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.439491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.439681 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:09:36.439865 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.439870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.440583 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.440584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.441407 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.441409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.441834 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.441835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.441972 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.442561 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:09:36.442569 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:09:36.444069 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:09:36.445918 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:09:36.445924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:09:36.445926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:09:36.446206 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:09:36.446428 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:09:36.446429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:09:36.446429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:09:36.446548 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:09:36.446831 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:09:36.446836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:09:36.446982 (+ 146us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.447734 (+ 752us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.447886 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260927 11:09:36.448547 29476 heartbeater.cc:347] Connected to a master server at 127.17.95.254:35301
I20260927 11:09:36.448660 29476 heartbeater.cc:464] Registering TS with master...
I20260927 11:09:36.448841 29476 heartbeater.cc:511] Master 127.17.95.254:35301 requested a full tablet report, sending...
I20260927 11:09:36.449234 28992 ts_manager.cc:194] Registered new tserver with Master: 30d76b0e4f4e475590d9407c693ece1b (127.17.95.195:33525)
I20260927 11:09:36.449575 28992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:50399
I20260927 11:09:36.450505 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:09:36.458642 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.452040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.17.95.254:35301)
0927 11:09:36.452077 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.452080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.452231 (+ 151us) server_negotiation.cc:407] Connection header received
0927 11:09:36.452282 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.452283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.452292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.452343 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:09:36.452891 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.453350 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.454127 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.454300 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.454388 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.455792 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:09:36.455793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:09:36.455793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:09:36.455799 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:09:36.457357 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:36.457809 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:36.457810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:36.457811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:36.457860 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:09:36.458048 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:09:36.458049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:09:36.458049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:09:36.458163 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:09:36.458243 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.458472 (+ 229us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.458559 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:09:36.460660 28992 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:09:36.462815 28992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:09:36.492341 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.476816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:38435 (local address 127.0.0.1:46112)
0927 11:09:36.487142 (+ 10326us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.487154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.487210 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.487570 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.487582 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.487584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:09:36.487626 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.487627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.488811 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.488812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.489868 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.489871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.492052 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.492053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.492132 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.492144 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.492208 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.492256 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10307,"thread_start_us":76,"threads_started":1}
I20260927 11:09:36.494093 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.473662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:39185 (local address 127.0.0.1:57900)
0927 11:09:36.477912 (+ 4250us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.477922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.477965 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.479316 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.479317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.479318 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:09:36.479345 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.479346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.480646 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.480647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.484801 (+ 4154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.484804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.493861 (+ 9057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.493862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.493938 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.493942 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.493979 (+ 37us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.494019 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4235,"thread_start_us":75,"threads_started":1}
I20260927 11:09:36.495628 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.473397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:33525 (local address 127.0.0.1:45484)
0927 11:09:36.477163 (+ 3766us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.477205 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.477699 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.479167 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.479170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.479172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:09:36.479250 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.479251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.485564 (+ 6313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.485565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.492727 (+ 7162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.492729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.495397 (+ 2668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.495398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.495466 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.495469 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.495507 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.495558 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3734,"thread_start_us":89,"threads_started":1}
I20260927 11:09:36.496932 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.473764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.17.95.193:39185)
0927 11:09:36.477860 (+ 4096us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.477866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.478019 (+ 153us) server_negotiation.cc:407] Connection header received
0927 11:09:36.478058 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.478060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.478078 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.478142 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:09:36.479417 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.480526 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.484895 (+ 4369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.493401 (+ 8506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.493475 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.493549 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.496815 (+ 3266us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.496864 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4059,"thread_start_us":62,"threads_started":1}
I20260927 11:09:36.497718 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.473893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.17.95.195:33525)
0927 11:09:36.478518 (+ 4625us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.478523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.478546 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:09:36.478582 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.478584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.478599 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.478665 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:09:36.483513 (+ 4848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.485454 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.494445 (+ 8991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.495227 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.495296 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.495363 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.497600 (+ 2237us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.497640 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4589,"thread_start_us":73,"threads_started":1}
I20260927 11:09:36.497843 29135 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbb7a565e9943a0b2a53601990aabc5 (DEFAULT_TABLE table=test-table [id=18778bca3d584b28a8f43eb25f5417a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:09:36.498260 29135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbb7a565e9943a0b2a53601990aabc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:09:36.498483 29410 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbb7a565e9943a0b2a53601990aabc5 (DEFAULT_TABLE table=test-table [id=18778bca3d584b28a8f43eb25f5417a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:09:36.498811 29410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbb7a565e9943a0b2a53601990aabc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:09:36.504925 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.476949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.17.95.194:38435)
0927 11:09:36.487359 (+ 10410us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.487363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.487373 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:09:36.487410 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.487412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.487428 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.487496 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:09:36.487694 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.488715 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.491157 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.491881 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.491950 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.492020 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.504755 (+ 12735us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.504843 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10369,"thread_start_us":76,"threads_started":1}
I20260927 11:09:36.505580 29275 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbb7a565e9943a0b2a53601990aabc5 (DEFAULT_TABLE table=test-table [id=18778bca3d584b28a8f43eb25f5417a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:09:36.505962 29275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbb7a565e9943a0b2a53601990aabc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:09:36.507270 29494 tablet_bootstrap.cc:492] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Bootstrap starting.
I20260927 11:09:36.508278 29494 tablet_bootstrap.cc:654] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Neither blocks nor log segments found. Creating new log.
I20260927 11:09:36.508563 29494 log.cc:824] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Log is configured to *not* fsync() on all Append() calls
I20260927 11:09:36.511143 29497 tablet_bootstrap.cc:492] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: Bootstrap starting.
I20260927 11:09:36.512310 29497 tablet_bootstrap.cc:654] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: Neither blocks nor log segments found. Creating new log.
I20260927 11:09:36.512771 29497 log.cc:824] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: Log is configured to *not* fsync() on all Append() calls
I20260927 11:09:36.513157 29495 tablet_bootstrap.cc:492] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a: Bootstrap starting.
I20260927 11:09:36.513935 29494 tablet_bootstrap.cc:492] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: No bootstrap required, opened a new log
I20260927 11:09:36.514006 29494 ts_tablet_manager.cc:1397] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260927 11:09:36.515947 29494 raft_consensus.cc:348] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.516129 29494 raft_consensus.cc:374] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:09:36.516168 29494 raft_consensus.cc:729] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d76b0e4f4e475590d9407c693ece1b, State: Initialized, Role: FOLLOWER
I20260927 11:09:36.516295 29494 consensus_queue.cc:260] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.516541 29494 ts_tablet_manager.cc:1428] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:09:36.516655 29476 heartbeater.cc:503] Master 127.17.95.254:35301 was elected leader, sending a full tablet report...
I20260927 11:09:36.517776 29495 tablet_bootstrap.cc:654] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a: Neither blocks nor log segments found. Creating new log.
I20260927 11:09:36.518558 29495 log.cc:824] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a: Log is configured to *not* fsync() on all Append() calls
I20260927 11:09:36.519383 29497 tablet_bootstrap.cc:492] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: No bootstrap required, opened a new log
I20260927 11:09:36.519493 29497 ts_tablet_manager.cc:1397] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260927 11:09:36.521927 29497 raft_consensus.cc:348] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.522246 29497 raft_consensus.cc:374] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:09:36.522359 29497 raft_consensus.cc:729] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d3d832e5c014f91a7ea69fa47cd3c1e, State: Initialized, Role: FOLLOWER
I20260927 11:09:36.522547 29497 consensus_queue.cc:260] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.523245 29497 ts_tablet_manager.cc:1428] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260927 11:09:36.523406 29341 heartbeater.cc:503] Master 127.17.95.254:35301 was elected leader, sending a full tablet report...
I20260927 11:09:36.525503 29495 tablet_bootstrap.cc:492] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a: No bootstrap required, opened a new log
I20260927 11:09:36.526017 29495 ts_tablet_manager.cc:1397] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260927 11:09:36.528684 29495 raft_consensus.cc:348] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.528893 29495 raft_consensus.cc:374] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:09:36.528973 29495 raft_consensus.cc:729] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7e0167e1cc442596974b3b4e83cd0a, State: Initialized, Role: FOLLOWER
I20260927 11:09:36.529171 29495 consensus_queue.cc:260] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.529490 29495 ts_tablet_manager.cc:1428] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:09:36.529665 29204 heartbeater.cc:503] Master 127.17.95.254:35301 was elected leader, sending a full tablet report...
I20260927 11:09:36.561564 29499 raft_consensus.cc:482] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:09:36.561697 29499 raft_consensus.cc:504] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.562104 29499 leader_election.cc:288] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d3d832e5c014f91a7ea69fa47cd3c1e (127.17.95.194:38435), 8e7e0167e1cc442596974b3b4e83cd0a (127.17.95.193:39185)
I20260927 11:09:36.566728 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.562329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:38435 (local address 127.17.95.195:45919)
0927 11:09:36.562423 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.562434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.562473 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.562759 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.562760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.562762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:09:36.562828 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.562830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.564156 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.564158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.565375 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.565379 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.566494 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.566495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.566576 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.566579 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.566623 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.566658 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260927 11:09:36.567207 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.562566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:45919 (local address 127.17.95.194:38435)
0927 11:09:36.562607 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.562609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.562616 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:09:36.562642 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.562642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.562652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.562696 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:09:36.563250 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.564035 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.565855 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.566393 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.566455 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.566464 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.567109 (+ 645us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.567145 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:09:36.567512 29291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbb7a565e9943a0b2a53601990aabc5" candidate_uuid: "30d76b0e4f4e475590d9407c693ece1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" is_pre_election: true
I20260927 11:09:36.567662 29291 raft_consensus.cc:2454] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d76b0e4f4e475590d9407c693ece1b in term 0.
I20260927 11:09:36.567888 29364 leader_election.cc:302] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d76b0e4f4e475590d9407c693ece1b, 8d3d832e5c014f91a7ea69fa47cd3c1e; no voters:
I20260927 11:09:36.568012 29499 raft_consensus.cc:2781] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:09:36.568078 29499 raft_consensus.cc:482] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:09:36.568107 29499 raft_consensus.cc:3037] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:09:36.568948 29499 raft_consensus.cc:504] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.569080 29499 leader_election.cc:288] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [CANDIDATE]: Term 1 election: Requested vote from peers 8d3d832e5c014f91a7ea69fa47cd3c1e (127.17.95.194:38435), 8e7e0167e1cc442596974b3b4e83cd0a (127.17.95.193:39185)
I20260927 11:09:36.569298 29291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbb7a565e9943a0b2a53601990aabc5" candidate_uuid: "30d76b0e4f4e475590d9407c693ece1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e"
I20260927 11:09:36.569368 29291 raft_consensus.cc:3037] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:09:36.570165 29291 raft_consensus.cc:2454] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d76b0e4f4e475590d9407c693ece1b in term 1.
I20260927 11:09:36.570318 29364 leader_election.cc:302] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d76b0e4f4e475590d9407c693ece1b, 8d3d832e5c014f91a7ea69fa47cd3c1e; no voters:
I20260927 11:09:36.570392 29499 raft_consensus.cc:2781] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:09:36.570523 29499 raft_consensus.cc:686] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 1 LEADER]: Becoming Leader. State: Replica: 30d76b0e4f4e475590d9407c693ece1b, State: Running, Role: LEADER
I20260927 11:09:36.570611 29499 consensus_queue.cc:237] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } }
I20260927 11:09:36.571446 28991 catalog_manager.cc:5495] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d76b0e4f4e475590d9407c693ece1b (127.17.95.195). New cstate: current_term: 1 leader_uuid: "30d76b0e4f4e475590d9407c693ece1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d76b0e4f4e475590d9407c693ece1b" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 33525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 } health_report { overall_health: UNKNOWN } } }
I20260927 11:09:36.574793 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.562973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:39185 (local address 127.17.95.195:36281)
0927 11:09:36.566800 (+ 3827us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.566812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:09:36.566853 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:09:36.571805 (+ 4952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:09:36.571807 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:09:36.571808 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:09:36.571837 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.571838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.572737 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.572738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.573797 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:09:36.573799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.574595 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.574596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:09:36.574653 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.574656 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:09:36.574699 (+ 43us) client_negotiation.cc:240] Negotiation successful
0927 11:09:36.574729 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3817,"thread_start_us":2826,"threads_started":1}
I20260927 11:09:36.575086 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.563174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:36281 (local address 127.17.95.193:39185)
0927 11:09:36.563215 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.563217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.566902 (+ 3685us) server_negotiation.cc:407] Connection header received
0927 11:09:36.566927 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.566928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.566937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.566975 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:09:36.571915 (+ 4940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.572639 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.573887 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.574498 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.574559 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.574567 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.574951 (+ 384us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.574994 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:09:36.575346 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbb7a565e9943a0b2a53601990aabc5" candidate_uuid: "30d76b0e4f4e475590d9407c693ece1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" is_pre_election: true
I20260927 11:09:36.575510 29158 raft_consensus.cc:2454] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d76b0e4f4e475590d9407c693ece1b in term 0.
I20260927 11:09:36.575650 29157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbb7a565e9943a0b2a53601990aabc5" candidate_uuid: "30d76b0e4f4e475590d9407c693ece1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e7e0167e1cc442596974b3b4e83cd0a"
I20260927 11:09:36.575726 29157 raft_consensus.cc:3037] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:09:36.576648 29157 raft_consensus.cc:2454] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d76b0e4f4e475590d9407c693ece1b in term 1.
I20260927 11:09:36.594455 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.591328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.17.95.195:33525)
0927 11:09:36.591363 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:09:36.591365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:09:36.591371 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:09:36.591401 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:09:36.591402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:09:36.591418 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:09:36.591473 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:09:36.591662 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.592347 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.593423 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:09:36.593734 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:09:36.593788 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:09:36.593980 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:09:36.594092 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:09:36.594156 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:09:36.594353 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 11:09:36.594370 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:09:36.596156 29157 raft_consensus.cc:1261] T 6dbb7a565e9943a0b2a53601990aabc5 P 8e7e0167e1cc442596974b3b4e83cd0a [term 1 FOLLOWER]: Refusing update from remote peer 30d76b0e4f4e475590d9407c693ece1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:09:36.596156 29291 raft_consensus.cc:1261] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 1 FOLLOWER]: Refusing update from remote peer 30d76b0e4f4e475590d9407c693ece1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:09:36.596468 29499 consensus_queue.cc:1035] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d3d832e5c014f91a7ea69fa47cd3c1e" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:09:36.598470 29499 consensus_queue.cc:1035] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e7e0167e1cc442596974b3b4e83cd0a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:09:36.601807 29507 mvcc.cc:205] Tried to move back new op lower bound from 7333918214535589888 to 7333918214433804288. Current Snapshot: MvccSnapshot[applied={T|T < 7333918214535589888}]
I20260927 11:09:36.602262 29508 mvcc.cc:205] Tried to move back new op lower bound from 7333918214535589888 to 7333918214433804288. Current Snapshot: MvccSnapshot[applied={T|T < 7333918214535589888}]
I20260927 11:09:36.603850 29510 mvcc.cc:205] Tried to move back new op lower bound from 7333918214535589888 to 7333918214433804288. Current Snapshot: MvccSnapshot[applied={T|T < 7333918214535589888}]
I20260927 11:09:36.606678 28992 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:09:36.606762 28992 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:09:36.607402 28992 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 375815d0a5af4034931af1bc5a7ad74b: Sending DeleteTablet for 3 replicas of tablet 6dbb7a565e9943a0b2a53601990aabc5
I20260927 11:09:36.607971 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 29073
I20260927 11:09:36.608656 29275 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dbb7a565e9943a0b2a53601990aabc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:09:36 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:46112
I20260927 11:09:36.609108 29410 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dbb7a565e9943a0b2a53601990aabc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:09:36 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:45484
I20260927 11:09:36.609489 29517 tablet_replica.cc:330] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e: stopping tablet replica
I20260927 11:09:36.609604 29517 raft_consensus.cc:2229] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:09:36.609705 29517 raft_consensus.cc:2258] T 6dbb7a565e9943a0b2a53601990aabc5 P 8d3d832e5c014f91a7ea69fa47cd3c1e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 11:09:36.614066 28979 connection.cc:535] client connection to 127.17.95.193:39185 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 11:09:36.614171 28979 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:09:36.614642 29518 tablet_replica.cc:330] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: stopping tablet replica
I20260927 11:09:36.614732 29518 raft_consensus.cc:2229] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:09:36.614861 29518 raft_consensus.cc:2258] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:09:36.615348 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.615142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:39185 (local address 127.0.0.1:57902)
0927 11:09:36.615278 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.615310 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260927 11:09:36.615456 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 29210
W20260927 11:09:36.615463 28979 catalog_manager.cc:4584] TS 8e7e0167e1cc442596974b3b4e83cd0a (127.17.95.193:39185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dbb7a565e9943a0b2a53601990aabc5: Network error: Client connection negotiation failed: client connection to 127.17.95.193:39185: connect: Connection refused (error 111)
I20260927 11:09:36.616133 29518 ts_tablet_manager.cc:1905] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:09:36.618047 29518 ts_tablet_manager.cc:1918] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:09:36.618129 29518 log.cc:1196] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/6dbb7a565e9943a0b2a53601990aabc5
I20260927 11:09:36.618409 29518 ts_tablet_manager.cc:1939] T 6dbb7a565e9943a0b2a53601990aabc5 P 30d76b0e4f4e475590d9407c693ece1b: Deleting consensus metadata
I20260927 11:09:36.618781 28979 catalog_manager.cc:4841] TS 30d76b0e4f4e475590d9407c693ece1b (127.17.95.195:33525): tablet 6dbb7a565e9943a0b2a53601990aabc5 (table test-table [id=18778bca3d584b28a8f43eb25f5417a9]) successfully deleted
I20260927 11:09:36.621969 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:09:36.621875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:38435 (local address 127.0.0.1:46116)
0927 11:09:36.621913 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0927 11:09:36.621935 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.194:38435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260927 11:09:36.622001 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 29347
I20260927 11:09:36.627372 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 28960
I20260927 11:09:36.633968 17791 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 11:09:36.967787 17791 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 11:09:36.967869 17791 mini_ranger.cc:66] Stopping Ranger...
I20260927 11:09:37.311940 17791 mini_ranger.cc:68] Stopped Ranger
2026-09-27 11:09:37.312 UTC [28546] LOG: received smart shutdown request
2026-09-27 11:09:37.315 UTC [28546] LOG: background worker "logical replication launcher" (PID 28555) exited with exit code 1
2026-09-27 11:09:37.315 UTC [28550] LOG: shutting down
2026-09-27 11:09:37.458 UTC [28546] LOG: database system is shut down
2026-09-27T11:09:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43533 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:09:37 dist-test-slave-6j0v krb5kdc[29528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:09:37 dist-test-slave-6j0v krb5kdc[29528](info): set up 2 sockets
Sep 27 11:09:37 dist-test-slave-6j0v krb5kdc[29528](info): commencing operation
krb5kdc: starting...
W20260927 11:09:38.570909 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:09:38 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:09:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/rangeradmin_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/rangeradmin_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/rangerlookup_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/rangerlookup_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
I20260927 11:09:38.725349 17791 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWCkVYM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/postgres -l logfile start
2026-09-27 11:09:41.133 UTC [29558] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 11:09:41.133 UTC [29558] LOG: listening on IPv4 address "127.17.95.212", port 38693
2026-09-27 11:09:41.137 UTC [29558] LOG: listening on Unix socket "/tmp/.s.PGSQL.38693"
2026-09-27 11:09:41.171 UTC [29561] LOG: database system was shut down at 2026-09-27 11:09:39 UTC
2026-09-27 11:09:41.175 UTC [29558] LOG: database system is ready to accept connections
I20260927 11:09:42.183038 17791 mini_postgres.cc:96] Postgres bound to 38693
2026-09-27 11:09:42.189 UTC [29569] FATAL: database "slave" does not exist
127.17.95.212:38693 - accepting connections
I20260927 11:09:42.190012 17791 mini_ranger.cc:161] Starting Ranger...
I20260927 11:09:42.203905 17791 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 11:09:42.660707 17791 mini_ranger.cc:87] Created ranger Postgres database
I20260927 11:09:42.660791 17791 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-admin
2026-09-27 11:09:42,927 [I] DB FLAVOR :POSTGRES
2026-09-27 11:09:42,927 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 11:09:42,928 [I] Checking connection..
2026-09-27 11:09:42,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 11:09:43,190 [I] Checking connection passed.
2026-09-27 11:09:43,190 [I] --------- Verifying version history table ---------
2026-09-27 11:09:43,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 11:09:43,453 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 11:09:43,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 11:09:43,699 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 11:09:43,699 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 11:09:43,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 11:09:43.918 UTC [29633] WARNING: there is no transaction in progress
2026-09-27 11:09:43,924 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 11:09:43,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 11:09:44,179 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 11:09:44,180 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 11:09:44,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 11:09:44,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 11:09:44,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6j0v.c.gcp-upstream.internal','N') ;"
2026-09-27 11:09:44,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 11:09:45,100 [I] Table x_portal_user does not exist in database ranger
2026-09-27 11:09:45,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 11:09:45,324 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 11:09:45,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 11:09:45,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 11:09:45,814 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 11:09:45,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 11:09:46.739 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.756 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.772 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.787 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.802 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.819 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.883 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.890 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.903 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.913 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.923 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.935 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.945 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.951 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.958 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:46.964 UTC [29741] WARNING: there is no transaction in progress
2026-09-27 11:09:47,567 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 11:09:47,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 11:09:47,790 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 11:09:47,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 11:09:48,039 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 11:09:48,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 11:09:48,290 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 11:09:48,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 11:09:48,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 11:09:48,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6j0v.c.gcp-upstream.internal';"
2026-09-27 11:09:49,015 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 11:09:49,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 11:09:49,260 [I] DB_PATCHES have already been applied
I20260927 11:09:49.265390 17791 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWCkVYM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 11:09:49.265447 17791 mini_ranger.cc:193] Using host: 127.17.95.212
I20260927 11:09:49.268371 17791 mini_ranger.cc:227] Ranger admin URL: http://127.17.95.212:34287
Sep 27, 2026 11:09:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 11:09:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 11:09:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 11:09:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 11:09:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 11:09:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 11:09:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 11:09:50.390818 17791 mini_ranger.cc:160] Time spent starting Ranger: real 8.201s user 0.001s sys 0.007s
Sep 27, 2026 11:10:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 11:10:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 27, 2026 11:10:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 11:10:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 11:10:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 11:10:10.421058 17791 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.95.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.95.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/rangerkms_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.95.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/rangerkms_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.95.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.95.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.95.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/HTTP_127.17.95.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 11:10:10 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507410, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 11:10:10.519248 17791 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 11:10:10.529140 17791 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 11:10:11.346547 17791 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 11:10:11.346627 17791 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms
I20260927 11:10:11.346642 17791 mini_ranger_kms.cc:227] Using postgres at 127.17.95.212:38693
2026-09-27 11:10:11,514 [I] DB FLAVOR :POSTGRES
2026-09-27 11:10:11,514 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 11:10:11,514 [I] Checking connection
2026-09-27 11:10:11,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 11:10:11,753 [I] connection success
2026-09-27 11:10:11,753 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 11:10:11,753 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 11:10:11,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 11:10:12,008 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 11:10:12,009 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 11:10:12,009 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 11:10:12,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.95.212:38693/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 11:10:12,263 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 11:10:12.703387 17791 mini_ranger_kms.cc:310] Created kms service
I20260927 11:10:12.889155 17791 mini_ranger_kms.cc:326] Created kudu user
I20260927 11:10:12.931272 17791 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 11:10:12 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507410, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:13.318516 17791 mini_ranger_kms.cc:384] Added ranger policy
I20260927 11:10:13.319053 17791 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWCkVYM/build/release/bin/postgresql.jar:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWCkVYM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 11:10:13.319106 17791 mini_ranger_kms.cc:240] Using host: 127.17.95.212
I20260927 11:10:13.322443 17791 mini_ranger_kms.cc:276] Ranger KMS PID: 29921
I20260927 11:10:13.322510 17791 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.95.212:48901
Sep 27, 2026 11:10:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 11:10:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 11:10:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 11:10:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 11:10:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 11:10:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 27 11:10:15 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790507415, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.95.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:15.526515 17791 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.007s user 0.000s sys 0.151s
I20260927 11:10:15.526633 17791 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 11:10:15.526686 17791 mini_ranger_kms.cc:401] 127.17.95.212:48901/kms/v1/keys
Sep 27 11:10:15 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507415, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.95.212@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
Sep 27 11:10:16 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507415, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.95.212@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
Sep 27 11:10:18 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:10:18.123616 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33605
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:33605
--ranger_config_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:18.222445 29972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:18.222683 29972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:18.222715 29972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:18.223879 29972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:18.223935 29972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:18.223956 29972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:18.223974 29972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:18.224001 29972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:18.225417 29972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.95.212:48901/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:33605
--ranger_config_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:18.225665 29972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:18.225863 29972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:18.231356 29980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:18.231381 29978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:18.231353 29977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:18.231678 29972 server_base.cc:1034] running on GCE node
I20260927 11:10:18.231974 29972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:18.232311 29972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:18.233493 29972 hybrid_clock.cc:648] HybridClock initialized: now 1790507418233476 us; error 25 us; skew 500 ppm
Sep 27 11:10:18 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507418, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:18.236272 29972 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:18.236716 29972 webserver.cc:460] Webserver started at http://127.17.95.254:42867/ using document root <none> and password file <none>
I20260927 11:10:18.236922 29972 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:18.236980 29972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:18.237138 29972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:10:18 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507418, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:18.299046 29972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "0f835cc839fd4cbbab7035b9d6802f09"
format_stamp: "Formatted at 2026-09-27 11:10:18 on dist-test-slave-6j0v"
server_key: "7cc862f95e74e151292c531da89c0c01"
server_key_iv: "d3bd6b3d392539a5d95e331ac32277f9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:18.299503 29972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "0f835cc839fd4cbbab7035b9d6802f09"
format_stamp: "Formatted at 2026-09-27 11:10:18 on dist-test-slave-6j0v"
server_key: "7cc862f95e74e151292c531da89c0c01"
server_key_iv: "d3bd6b3d392539a5d95e331ac32277f9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:18.300936 29972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:10:18.312307 29987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:18.312470 29972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:18.312568 29972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "0f835cc839fd4cbbab7035b9d6802f09"
format_stamp: "Formatted at 2026-09-27 11:10:18 on dist-test-slave-6j0v"
server_key: "7cc862f95e74e151292c531da89c0c01"
server_key_iv: "d3bd6b3d392539a5d95e331ac32277f9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:18.312660 29972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:18.329319 29972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:18.331476 29972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:18.331614 29972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:18.336212 29972 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:33605
I20260927 11:10:18.336287 30039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:33605 every 8 connection(s)
I20260927 11:10:18.336637 29972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:18.337292 30040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:18.339490 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 29972
I20260927 11:10:18.339519 30040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Bootstrap starting.
I20260927 11:10:18.339596 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:10:18.339781 17791 external_mini_cluster.cc:1428] Setting key 56e248d3745ecb7b0306793782b6262b
I20260927 11:10:18.340234 30040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:18.340656 30040 log.cc:824] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:18.341457 30040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: No bootstrap required, opened a new log
I20260927 11:10:18.343046 30040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:18.343257 30040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:18.343320 30040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f835cc839fd4cbbab7035b9d6802f09, State: Initialized, Role: FOLLOWER
I20260927 11:10:18.343469 30040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:18.343575 30040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:18.343624 30040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:18.343686 30040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:18.344226 30040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:18.344331 30040 leader_election.cc:302] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f835cc839fd4cbbab7035b9d6802f09; no voters:
I20260927 11:10:18.344527 30040 leader_election.cc:288] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:10:18.344589 30045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:18.344815 30045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 LEADER]: Becoming Leader. State: Replica: 0f835cc839fd4cbbab7035b9d6802f09, State: Running, Role: LEADER
I20260927 11:10:18.344915 30045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:18.345202 30045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f835cc839fd4cbbab7035b9d6802f09. Latest consensus state: current_term: 1 leader_uuid: "0f835cc839fd4cbbab7035b9d6802f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } } }
I20260927 11:10:18.345268 30045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:18.345211 30047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f835cc839fd4cbbab7035b9d6802f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } } }
I20260927 11:10:18.345606 30047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:18.344830 30040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:18.347203 30040 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 11:10:18.347592 30049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 27 11:10:18 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:18.350472 30049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:18.355288 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:18.340295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.17.95.254:33605)
0927 11:10:18.340504 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:18.340513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:18.340556 (+ 43us) server_negotiation.cc:407] Connection header received
0927 11:10:18.341102 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:18.341111 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:18.341189 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:18.341346 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:18.342255 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:18.344088 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:18.345130 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:18.345775 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:18.345880 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:18.349352 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:18.349409 (+ 57us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:18.349409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:18.349514 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:18.352660 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:18.353102 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:18.353108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:18.353108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:18.353225 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:18.353462 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:18.353463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:18.353463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:18.353879 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:18.354056 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:18.354620 (+ 564us) server_negotiation.cc:299] Negotiation successful
0927 11:10:18.354923 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260927 11:10:18.362226 30049 catalog_manager.cc:1347] Generated new cluster ID: 50800e2b6a1844569beb1800100e1b84
I20260927 11:10:18.362275 30049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:10:18.377132 30049 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:10:18.377936 30049 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:18.393822 30049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Generated new TSK 0
I20260927 11:10:18.394071 30049 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:10:19.078537 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33605
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:19.196209 30085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:19.196491 30085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:19.196520 30085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:19.198172 30085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:19.198230 30085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:19.198267 30085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:10:19.200338 30085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.95.212:48901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:19.200587 30085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:19.200856 30085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:19.206771 30090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:19.206804 30093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:19.206776 30091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:19.207733 30085 server_base.cc:1034] running on GCE node
I20260927 11:10:19.207983 30085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:19.208236 30085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:19.209439 30085 hybrid_clock.cc:648] HybridClock initialized: now 1790507419209397 us; error 35 us; skew 500 ppm
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:19.212392 30085 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:10:19.212934 30085 webserver.cc:460] Webserver started at http://127.17.95.193:37373/ using document root <none> and password file <none>
I20260927 11:10:19.213199 30085 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:19.213277 30085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:19.213490 30085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:19.231573 30085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "d6cc229d010a4240a0290988f1fabfe0"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "e531045bbdbad90527444e540fed9fd5"
server_key_iv: "aa4e0feabf726c2bf9fdf5595f6fce5f"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.232318 30085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "d6cc229d010a4240a0290988f1fabfe0"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "e531045bbdbad90527444e540fed9fd5"
server_key_iv: "aa4e0feabf726c2bf9fdf5595f6fce5f"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.235152 30085 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 11:10:19.249852 30100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.250072 30085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:19.250169 30085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "d6cc229d010a4240a0290988f1fabfe0"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "e531045bbdbad90527444e540fed9fd5"
server_key_iv: "aa4e0feabf726c2bf9fdf5595f6fce5f"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.250273 30085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:19.275265 30085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:19.277709 30085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:19.277915 30085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:19.278234 30085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:19.278640 30085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:19.278743 30085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.278836 30085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:19.278901 30085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.285483 30085 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:45415
I20260927 11:10:19.285969 30085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:10:19.287458 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30085
I20260927 11:10:19.287570 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:10:19.287756 17791 external_mini_cluster.cc:1428] Setting key cf1b2e719790f32f0d6e647e25c7b5ff
I20260927 11:10:19.292160 30215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:45415 every 8 connection(s)
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
I20260927 11:10:19.304065 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:19.292817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33605 (local address 127.17.95.193:37155)
0927 11:10:19.293523 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:19.293583 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:19.294216 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:19.294667 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:19.294671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:19.294920 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:19.295226 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:19.295234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.296247 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.296248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:19.297523 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:19.297526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.298088 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.298089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:19.298224 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:19.298825 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:19.298835 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:19.300522 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:19.302477 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:19.302483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:19.302485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:19.302785 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:19.302998 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:19.302999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:19.302999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:19.303137 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:19.303413 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:19.303420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:19.303595 (+ 175us) client_negotiation.cc:769] Sending connection context
0927 11:10:19.303696 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 11:10:19.303848 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":86,"threads_started":1}
I20260927 11:10:19.304775 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:19.293109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:37155 (local address 127.17.95.254:33605)
0927 11:10:19.294404 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:19.294405 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:19.294413 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:10:19.294437 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:19.294437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:19.294451 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:19.294516 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:19.295345 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.296139 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.297631 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.297941 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.298007 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:19.300640 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:19.300642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:19.300642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:19.300649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:19.302387 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:19.302869 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:19.302870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:19.302870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:19.302929 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:19.303205 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:19.303206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:19.303207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:19.303335 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:19.303389 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:19.304587 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0927 11:10:19.304674 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1266,"thread_start_us":87,"threads_started":1}
I20260927 11:10:19.305431 30216 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33605
I20260927 11:10:19.305598 30216 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:19.305842 30216 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:19.306437 30004 ts_manager.cc:194] Registered new tserver with Master: d6cc229d010a4240a0290988f1fabfe0 (127.17.95.193:45415)
I20260927 11:10:19.307376 30004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:37155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:10:19.326367 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33605
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:19.448571 30222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:19.448839 30222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:19.448877 30222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:19.450460 30222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:19.450515 30222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:19.450559 30222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:10:19.452734 30222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.95.212:48901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:19.453042 30222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:19.453343 30222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 11:10:19.459569 30232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:19.459635 30230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:19.459883 30229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:19.460249 30222 server_base.cc:1034] running on GCE node
I20260927 11:10:19.460462 30222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:19.460752 30222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:19.461944 30222 hybrid_clock.cc:648] HybridClock initialized: now 1790507419461922 us; error 33 us; skew 500 ppm
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:19.465342 30222 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:10:19.465924 30222 webserver.cc:460] Webserver started at http://127.17.95.194:34359/ using document root <none> and password file <none>
I20260927 11:10:19.466212 30222 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:19.466317 30222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:19.466485 30222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:19.503022 30222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "4184c84570404cda94cf5e4dcdcadde5"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "594d0dca5921c713cda311c16c918c8c"
server_key_iv: "4d3e3719869f6fcb23b332a92febb2c9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.503487 30222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "4184c84570404cda94cf5e4dcdcadde5"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "594d0dca5921c713cda311c16c918c8c"
server_key_iv: "4d3e3719869f6fcb23b332a92febb2c9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.505364 30222 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 11:10:19.524283 30239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.524523 30222 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260927 11:10:19.524613 30222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "4184c84570404cda94cf5e4dcdcadde5"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "594d0dca5921c713cda311c16c918c8c"
server_key_iv: "4d3e3719869f6fcb23b332a92febb2c9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.524693 30222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:19.537765 30222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:19.540060 30222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:19.540206 30222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:19.540472 30222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:19.540822 30222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:19.540874 30222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.540936 30222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:19.540964 30222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.548121 30222 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:43329
I20260927 11:10:19.548484 30222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:10:19.549160 30353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:43329 every 8 connection(s)
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:19.557832 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30222
I20260927 11:10:19.557938 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:10:19.557950 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:19.549199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33605 (local address 127.17.95.194:54551)
0927 11:10:19.549486 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:19.549530 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:19.550019 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:19.550259 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:19.550263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:19.550438 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:19.550620 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:19.550624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.551384 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.551385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:19.552149 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:19.552151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.552622 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.552623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:19.552749 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:19.553259 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:19.553268 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:19.554787 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:19.556433 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:19.556439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:19.556442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:19.556658 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:19.556877 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:19.556878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:19.556878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:19.556991 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:19.557266 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:19.557272 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:19.557407 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 11:10:19.557509 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 11:10:19.557646 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":70,"threads_started":1}
I20260927 11:10:19.558166 17791 external_mini_cluster.cc:1428] Setting key 736727e0730bed39e7893beb46bba6a6
I20260927 11:10:19.558459 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:19.549215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:54551 (local address 127.17.95.254:33605)
0927 11:10:19.549419 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:19.549421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:19.549613 (+ 192us) server_negotiation.cc:407] Connection header received
0927 11:10:19.550130 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:19.550131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:19.550146 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:19.550193 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:19.550721 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.551290 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.552241 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.552500 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.552585 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:19.554898 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:19.554900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:19.554900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:19.554919 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:19.556326 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:19.556748 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:19.556750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:19.556751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:19.556812 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:19.557074 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:19.557075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:19.557075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:19.557194 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:19.557263 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:19.557548 (+ 285us) server_negotiation.cc:299] Negotiation successful
0927 11:10:19.557637 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260927 11:10:19.561446 30354 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33605
I20260927 11:10:19.561573 30354 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:19.561748 30354 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:19.562166 30003 ts_manager.cc:194] Registered new tserver with Master: 4184c84570404cda94cf5e4dcdcadde5 (127.17.95.194:43329)
I20260927 11:10:19.562655 30003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:54551
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:10:19.588027 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33605
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:19.705859 30359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:19.706128 30359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:19.706167 30359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:19.707937 30359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:19.708062 30359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:19.708149 30359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:10:19.710316 30359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.95.212:48901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:19.710567 30359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:19.710862 30359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:19.716975 30367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:19.717324 30365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:19.717631 30359 server_base.cc:1034] running on GCE node
W20260927 11:10:19.717818 30364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:19.718075 30359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:19.718334 30359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:19.719515 30359 hybrid_clock.cc:648] HybridClock initialized: now 1790507419719491 us; error 36 us; skew 500 ppm
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:19.722420 30359 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:10:19.722924 30359 webserver.cc:460] Webserver started at http://127.17.95.195:46115/ using document root <none> and password file <none>
I20260927 11:10:19.723212 30359 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:19.723272 30359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:19.723398 30359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:19.735158 30359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "670ef349b0374d85b8736203531f36b3"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "caa04fc89d2f0b3460c1e4498f4e1d41"
server_key_iv: "e6d77638e6ae7b8f4a37bc3161b90f00"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.735668 30359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "670ef349b0374d85b8736203531f36b3"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "caa04fc89d2f0b3460c1e4498f4e1d41"
server_key_iv: "e6d77638e6ae7b8f4a37bc3161b90f00"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.743440 30359 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260927 11:10:19.764577 30374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.767912 30359 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.000s sys 0.000s
I20260927 11:10:19.768026 30359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "670ef349b0374d85b8736203531f36b3"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "caa04fc89d2f0b3460c1e4498f4e1d41"
server_key_iv: "e6d77638e6ae7b8f4a37bc3161b90f00"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.768126 30359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:19.780740 30359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:19.783005 30359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:19.783192 30359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:19.783437 30359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:19.783815 30359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:19.783896 30359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.783946 30359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:19.783968 30359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.790442 30359 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:43765
I20260927 11:10:19.790699 30487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:43765 every 8 connection(s)
I20260927 11:10:19.790913 30359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:10:19.795806 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30359
I20260927 11:10:19.795907 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:10:19.796087 17791 external_mini_cluster.cc:1428] Setting key e08a65e2b705211e4aebce63a564376b
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:19.802870 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:19.792404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:41651 (local address 127.17.95.254:33605)
0927 11:10:19.792448 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:19.792450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:19.792865 (+ 415us) server_negotiation.cc:407] Connection header received
0927 11:10:19.793496 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:19.793498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:19.793512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:19.793571 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:19.794339 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.795136 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.796617 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.796915 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.796957 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:19.799653 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:19.799654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:19.799654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:19.799660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:19.801300 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:19.801785 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:19.801786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:19.801786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:19.801857 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:19.802132 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:19.802133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:19.802133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:19.802268 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:19.802309 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:19.802692 (+ 383us) server_negotiation.cc:299] Negotiation successful
0927 11:10:19.802775 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:10:19.803503 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:19.792379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33605 (local address 127.17.95.195:41651)
0927 11:10:19.792711 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:19.792756 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:19.793378 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:19.793719 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:19.793723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:19.793963 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:19.794229 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:19.794235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.795234 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.795235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:19.796506 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:19.796508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:19.797061 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:19.797062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:19.797203 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:19.797895 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:19.797905 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:19.799455 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:19.801388 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:19.801394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:19.801396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:19.801667 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:19.801925 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:19.801926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:19.801926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:19.802055 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:19.802354 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:19.802359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:19.802527 (+ 168us) client_negotiation.cc:769] Sending connection context
0927 11:10:19.803131 (+ 604us) client_negotiation.cc:240] Negotiation successful
0927 11:10:19.803283 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":82,"threads_started":1}
I20260927 11:10:19.804112 30488 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33605
I20260927 11:10:19.804208 30488 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:19.804401 30488 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:19.804790 30003 ts_manager.cc:194] Registered new tserver with Master: 670ef349b0374d85b8736203531f36b3 (127.17.95.195:43765)
I20260927 11:10:19.805272 30003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:41651
I20260927 11:10:19.807786 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:10:19.808450 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30085
I20260927 11:10:19.815971 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30222
I20260927 11:10:19.821292 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30359
I20260927 11:10:19.826212 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 29972
I20260927 11:10:19.831081 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33605
--webserver_interface=127.17.95.254
--webserver_port=42867
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:33605
--ranger_config_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:19.913579 30491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:19.913764 30491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:19.913786 30491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:19.914978 30491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:19.915017 30491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:19.915031 30491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:19.915066 30491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:19.915110 30491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:19.916698 30491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:33605
--ranger_config_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=42867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:19.916927 30491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:19.917153 30491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:19.921876 30497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:19.922000 30491 server_base.cc:1034] running on GCE node
W20260927 11:10:19.921888 30499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:19.921888 30496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:19.922310 30491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:19.922641 30491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:19.923779 30491 hybrid_clock.cc:648] HybridClock initialized: now 1790507419923757 us; error 46 us; skew 500 ppm
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:19.926258 30491 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:19.926693 30491 webserver.cc:460] Webserver started at http://127.17.95.254:42867/ using document root <none> and password file <none>
I20260927 11:10:19.926903 30491 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:19.926954 30491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 11:10:19 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507419, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:19.980224 30491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:19.982308 30507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:19.982519 30491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 11:10:19.982631 30491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "0f835cc839fd4cbbab7035b9d6802f09"
format_stamp: "Formatted at 2026-09-27 11:10:18 on dist-test-slave-6j0v"
server_key: "7cc862f95e74e151292c531da89c0c01"
server_key_iv: "d3bd6b3d392539a5d95e331ac32277f9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:19.983124 30491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:20.006332 30491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:20.008579 30491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:20.008700 30491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:20.013016 30491 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:33605
I20260927 11:10:20.013063 30559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:33605 every 8 connection(s)
I20260927 11:10:20.013422 30491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:20.014608 30560 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:10:20.015311 30560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Bootstrap starting.
I20260927 11:10:20.019695 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30491
I20260927 11:10:20.020044 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:45415
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:33605
--webserver_port=37373
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:10:20.023667 30560 log.cc:824] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:20.025206 30560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:10:20.025489 30560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Bootstrap complete.
I20260927 11:10:20.027894 30560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:20.028117 30560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f835cc839fd4cbbab7035b9d6802f09, State: Initialized, Role: FOLLOWER
I20260927 11:10:20.028282 30560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:20.028350 30560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:20.028400 30560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:20.028450 30560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 1 FOLLOWER]: Advancing to term 2
I20260927 11:10:20.029405 30560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:20.029491 30560 leader_election.cc:302] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f835cc839fd4cbbab7035b9d6802f09; no voters:
I20260927 11:10:20.029667 30560 leader_election.cc:288] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 11:10:20.029732 30564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 11:10:20.029893 30564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [term 2 LEADER]: Becoming Leader. State: Replica: 0f835cc839fd4cbbab7035b9d6802f09, State: Running, Role: LEADER
I20260927 11:10:20.029924 30560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:20.030066 30564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } }
I20260927 11:10:20.030378 30566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f835cc839fd4cbbab7035b9d6802f09. Latest consensus state: current_term: 2 leader_uuid: "0f835cc839fd4cbbab7035b9d6802f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } } }
I20260927 11:10:20.030359 30565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0f835cc839fd4cbbab7035b9d6802f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f835cc839fd4cbbab7035b9d6802f09" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33605 } } }
I20260927 11:10:20.030465 30566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:20.030478 30565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:20.030719 30568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:20.031353 30568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:20.031545 30568 catalog_manager.cc:1259] Loaded cluster ID: 50800e2b6a1844569beb1800100e1b84
I20260927 11:10:20.031629 30568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:10:20.031783 30560 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 11:10:20.035388 30568 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:20.035894 30568 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Loaded TSK: 0
I20260927 11:10:20.036343 30568 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 11:10:20.106820 30562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:20.107026 30562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:20.107087 30562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:20.108247 30562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:20.108361 30562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:20.108445 30562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:10:20.110117 30562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:45415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=37373
--tserver_master_addrs=127.17.95.254:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:20.110343 30562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:20.110553 30562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:20.115479 30598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:20.115456 30600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:20.115861 30597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:20.115954 30562 server_base.cc:1034] running on GCE node
I20260927 11:10:20.116636 30562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:20.116881 30562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:20.118041 30562 hybrid_clock.cc:648] HybridClock initialized: now 1790507420118031 us; error 27 us; skew 500 ppm
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:20.120653 30562 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:10:20.121129 30562 webserver.cc:460] Webserver started at http://127.17.95.193:37373/ using document root <none> and password file <none>
I20260927 11:10:20.121357 30562 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:20.121439 30562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:20.144536 30562 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260927 11:10:20.145526 30608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.145692 30562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:20.145778 30562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "d6cc229d010a4240a0290988f1fabfe0"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "e531045bbdbad90527444e540fed9fd5"
server_key_iv: "aa4e0feabf726c2bf9fdf5595f6fce5f"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:20.146150 30562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:20.161319 30562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:20.163721 30562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:20.163884 30562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:20.164197 30562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:20.164605 30562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:20.164697 30562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.164798 30562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:20.164875 30562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.173224 30562 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:45415
I20260927 11:10:20.173637 30562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:10:20.174111 30721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:45415 every 8 connection(s)
I20260927 11:10:20.175393 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30562
I20260927 11:10:20.175683 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:43329
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:33605
--webserver_port=34359
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:20.187239 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.174580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33605 (local address 127.17.95.193:51721)
0927 11:10:20.175176 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:20.175228 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:20.175958 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:20.176669 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:20.176673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:20.176924 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:20.177259 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:20.177265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.178342 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.178343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:20.179162 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:20.179165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.180152 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.180154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:20.180286 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.180806 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:20.180816 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:20.182582 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:20.185120 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:20.185127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:20.185129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:20.185436 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:20.185718 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:20.185719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:20.185719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:20.185851 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:20.186361 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:20.186367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:20.186521 (+ 154us) client_negotiation.cc:769] Sending connection context
0927 11:10:20.186634 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 11:10:20.186934 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":81,"threads_started":1}
I20260927 11:10:20.187851 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.174645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:51721 (local address 127.17.95.254:33605)
0927 11:10:20.176165 (+ 1520us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:20.176174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:20.176209 (+ 35us) server_negotiation.cc:407] Connection header received
0927 11:10:20.176344 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:20.176350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:20.176402 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:20.176515 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:20.177382 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.178238 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.179262 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.179915 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.180019 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.182715 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:20.182719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:20.182719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:20.182732 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:20.185014 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.185532 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.185535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.185535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.185642 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.185928 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.185929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.185929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.186249 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:20.186331 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:20.187455 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0927 11:10:20.187582 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1463,"thread_start_us":78,"threads_started":1}
I20260927 11:10:20.188762 30722 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33605
I20260927 11:10:20.188885 30722 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.189121 30722 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
W20260927 11:10:20.190016 30722 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:33605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 11:10:20.190138 30722 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.190240 30722 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:20.190591 30722 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.190701 30722 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
W20260927 11:10:20.191058 30722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:10:20.280576 30726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:20.280786 30726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:20.280822 30726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:20.281919 30726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:20.281980 30726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:20.282021 30726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:10:20.283630 30726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:43329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=34359
--tserver_master_addrs=127.17.95.254:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:20.283947 30726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:20.284189 30726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:20.291602 30731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:20.291786 30732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:20.291817 30734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:20.292807 30726 server_base.cc:1034] running on GCE node
I20260927 11:10:20.292966 30726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:20.293177 30726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:20.294332 30726 hybrid_clock.cc:648] HybridClock initialized: now 1790507420294318 us; error 52 us; skew 500 ppm
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:20.297326 30726 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:10:20.297885 30726 webserver.cc:460] Webserver started at http://127.17.95.194:34359/ using document root <none> and password file <none>
I20260927 11:10:20.298183 30726 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:20.298312 30726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:20.328943 30726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:20.330139 30742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.330364 30726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:20.330454 30726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "4184c84570404cda94cf5e4dcdcadde5"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "594d0dca5921c713cda311c16c918c8c"
server_key_iv: "4d3e3719869f6fcb23b332a92febb2c9"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:20.330830 30726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:20.364375 30726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:20.366567 30726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:20.366740 30726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:20.367050 30726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:20.367466 30726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:20.367503 30726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.367527 30726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:20.367568 30726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.374568 30726 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:43329
I20260927 11:10:20.374975 30726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:10:20.375443 30855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:43329 every 8 connection(s)
I20260927 11:10:20.375828 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30726
I20260927 11:10:20.376161 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.95.212:48901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:43765
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:33605
--webserver_port=46115
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:20.386282 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.376018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:59895 (local address 127.17.95.254:33605)
0927 11:10:20.376057 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:20.376059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:20.376069 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:10:20.376581 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:20.376583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:20.376599 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:20.376648 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:20.377253 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.378029 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.379545 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.379909 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.379961 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.382623 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:20.382624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:20.382625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:20.382632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:20.384518 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.385111 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.385113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.385113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.385171 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.385474 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.385475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.385475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.385615 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:20.385662 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:20.386071 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 11:10:20.386157 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:10:20.387145 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.375733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33605 (local address 127.17.95.194:59895)
0927 11:10:20.375876 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:20.375916 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:20.376466 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:20.376715 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:20.376719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:20.376963 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:20.377148 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:20.377152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.378142 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.378143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:20.379448 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:20.379451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.380076 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.380077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:20.380237 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.380791 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:20.380801 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:20.382504 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:20.384614 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:20.384620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:20.384623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:20.385014 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:20.385246 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:20.385247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:20.385247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:20.385389 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:20.385731 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:20.385737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:20.385905 (+ 168us) client_negotiation.cc:769] Sending connection context
0927 11:10:20.386659 (+ 754us) client_negotiation.cc:240] Negotiation successful
0927 11:10:20.386856 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260927 11:10:20.387921 30856 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33605
I20260927 11:10:20.388056 30856 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.388238 30856 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
W20260927 11:10:20.388993 30856 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:33605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 11:10:20.389061 30856 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.389118 30856 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:20.389382 30856 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.389452 30856 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
W20260927 11:10:20.389681 30856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:10:20.473668 30859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:20.473938 30859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:20.473977 30859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:20.475652 30859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:20.475705 30859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:20.475744 30859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:10:20.477778 30859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.95.212:48901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:43765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=46115
--tserver_master_addrs=127.17.95.254:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:20.477996 30859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:20.478251 30859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:20.484263 30864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:20.484275 30867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:20.484694 30865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:20.484851 30859 server_base.cc:1034] running on GCE node
I20260927 11:10:20.485042 30859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:20.485316 30859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:20.486516 30859 hybrid_clock.cc:648] HybridClock initialized: now 1790507420486480 us; error 48 us; skew 500 ppm
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:20.489635 30859 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:10:20.490221 30859 webserver.cc:460] Webserver started at http://127.17.95.195:46115/ using document root <none> and password file <none>
I20260927 11:10:20.490460 30859 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:20.490537 30859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:20.511497 30859 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:20.512777 30875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.513000 30859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:20.513082 30859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "670ef349b0374d85b8736203531f36b3"
format_stamp: "Formatted at 2026-09-27 11:10:19 on dist-test-slave-6j0v"
server_key: "caa04fc89d2f0b3460c1e4498f4e1d41"
server_key_iv: "e6d77638e6ae7b8f4a37bc3161b90f00"
server_key_version: "kuduclusterkey@0"
I20260927 11:10:20.513542 30859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:20.534428 30859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:20.536448 30859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:20.536605 30859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:20.536859 30859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:20.537257 30859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:20.537308 30859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.537344 30859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:20.537370 30859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:20.548592 30859 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:43765
I20260927 11:10:20.549072 30859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:10:20.554563 30989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:43765 every 8 connection(s)
I20260927 11:10:20.557467 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 30859
Sep 27 11:10:20 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:20.568542 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.558088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.17.95.254:33605)
0927 11:10:20.558214 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:20.558216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:20.558382 (+ 166us) server_negotiation.cc:407] Connection header received
0927 11:10:20.558451 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:20.558451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:20.558464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:20.558523 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:20.561529 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.562133 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.563121 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.563413 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.563502 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.564882 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:20.564884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:20.564884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:20.564891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:20.566407 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.566821 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.566822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.566823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.566901 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.567365 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.567367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.567367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.567512 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:20.567695 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:20.568321 (+ 626us) server_negotiation.cc:299] Negotiation successful
0927 11:10:20.568421 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260927 11:10:20.569617 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.555733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:34589 (local address 127.17.95.254:33605)
0927 11:10:20.555800 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:20.555803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:20.555813 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:10:20.556439 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:20.556441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:20.556453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:20.556535 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:20.557776 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.558545 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.560548 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.560858 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.560942 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.563939 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:20.563941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:20.563941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:20.563949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:20.565866 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.566914 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.566915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.566916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.566979 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:20.568907 (+ 1928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:20.568908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:20.568908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:20.569037 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:20.569109 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:20.569454 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 11:10:20.569531 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:10:20.569796 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:20.555367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:33605 (local address 127.17.95.195:34589)
0927 11:10:20.555576 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:20.555627 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:20.556279 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:20.556615 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:20.556620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:20.556882 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:20.557184 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:20.557192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.559022 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.559023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:20.560427 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:20.560431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:20.561071 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:20.561073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:20.561224 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:20.561919 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:20.561930 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:20.563803 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:20.566505 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:20.566512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:20.566514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:20.566822 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:20.567093 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:20.567094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:20.567095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:20.567247 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:20.569118 (+ 1871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:20.569125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:20.569286 (+ 161us) client_negotiation.cc:769] Sending connection context
0927 11:10:20.569402 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 11:10:20.569568 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260927 11:10:20.570257 30990 heartbeater.cc:347] Connected to a master server at 127.17.95.254:33605
I20260927 11:10:20.570350 30990 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.570542 30990 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
W20260927 11:10:20.571267 30990 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:33605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 11:10:20.571353 30990 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.571434 30990 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:20.571792 30990 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:20.571882 30990 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
W20260927 11:10:20.572233 30990 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 11:10:21 dist-test-slave-6j0v krb5kdc[29528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790507421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:21.191279 30722 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:21.191426 30722 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:21.191779 30524 ts_manager.cc:194] Registered new tserver with Master: d6cc229d010a4240a0290988f1fabfe0 (127.17.95.193:45415)
I20260927 11:10:21.192480 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:51721
I20260927 11:10:21.389890 30856 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:21.390031 30856 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:21.390311 30524 ts_manager.cc:194] Registered new tserver with Master: 4184c84570404cda94cf5e4dcdcadde5 (127.17.95.194:43329)
I20260927 11:10:21.390851 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:59895
Sep 27 11:10:21 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:21.572427 30990 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:21.572572 30990 heartbeater.cc:511] Master 127.17.95.254:33605 requested a full tablet report, sending...
I20260927 11:10:21.572844 30524 ts_manager.cc:194] Registered new tserver with Master: 670ef349b0374d85b8736203531f36b3 (127.17.95.195:43765)
I20260927 11:10:21.573324 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:34589
I20260927 11:10:21.577330 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:10:21.589252 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.578253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.17.95.254:33605)
0927 11:10:21.578526 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.578529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.578683 (+ 154us) server_negotiation.cc:407] Connection header received
0927 11:10:21.578704 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.578705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.578716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.578768 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:21.579546 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.580270 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.581263 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.581553 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.581611 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.586077 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:21.586078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:21.586078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:21.586086 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:21.587919 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:21.588287 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:21.588288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:21.588288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:21.588352 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:21.588508 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:21.588508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:21.588509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:21.588618 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:21.588703 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.589089 (+ 386us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.589160 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":61,"threads_started":1}
I20260927 11:10:21.591159 30524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:10:21.593341 30524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:10:21.606200 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.596304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:43329 (local address 127.0.0.1:41412)
0927 11:10:21.601515 (+ 5211us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.601525 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:21.601579 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:21.601876 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:21.601879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:21.601882 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:21.601955 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.601956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.603246 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.603247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.604840 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.604843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.605880 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.605881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.605967 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.605981 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 11:10:21.606055 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 11:10:21.606107 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5194,"thread_start_us":85,"threads_started":1}
I20260927 11:10:21.606660 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.596759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.17.95.194:43329)
0927 11:10:21.601387 (+ 4628us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.601393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.601651 (+ 258us) server_negotiation.cc:407] Connection header received
0927 11:10:21.601690 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.601693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.601709 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.601778 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:21.602033 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.603139 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.604938 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.605710 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.605782 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.605842 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.606546 (+ 704us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.606585 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4592,"thread_start_us":57,"threads_started":1}
I20260927 11:10:21.607627 30773 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6372bd453f4cfdb852dad61682d0d6 (DEFAULT_TABLE table=test-table [id=67673d753e16403ebf61f1f255bfef8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:21.607995 30773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6372bd453f4cfdb852dad61682d0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:21.611747 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.596510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:43765 (local address 127.0.0.1:57062)
0927 11:10:21.607036 (+ 10526us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.607046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:21.607141 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:21.607765 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:21.607765 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:21.607767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:21.607849 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.607851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.609181 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.609182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.610224 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.610227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.611247 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.611247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.611303 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.611306 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:21.611639 (+ 333us) client_negotiation.cc:240] Negotiation successful
0927 11:10:21.611676 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10513,"thread_start_us":72,"threads_started":1}
I20260927 11:10:21.612071 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.596604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.17.95.195:43765)
0927 11:10:21.601462 (+ 4858us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.601466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.607112 (+ 5646us) server_negotiation.cc:407] Connection header received
0927 11:10:21.607539 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.607543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.607560 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.607616 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:21.607938 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.609079 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.610325 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.611046 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.611125 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.611180 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.611956 (+ 776us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.611995 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4827,"thread_start_us":48,"threads_started":1}
I20260927 11:10:21.612968 30911 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6372bd453f4cfdb852dad61682d0d6 (DEFAULT_TABLE table=test-table [id=67673d753e16403ebf61f1f255bfef8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:21.613277 30911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6372bd453f4cfdb852dad61682d0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:21.616353 31020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.596245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45415 (local address 127.0.0.1:36630)
0927 11:10:21.596648 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.596687 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:21.596749 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:21.612804 (+ 16055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:21.612805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:21.612808 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:21.612855 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.612856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.614159 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.614160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.615214 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.615217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.616115 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.616116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.616173 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.616177 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:10:21.616239 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 11:10:21.616284 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":62,"threads_started":1}
I20260927 11:10:21.617117 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.596463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36630 (local address 127.17.95.193:45415)
0927 11:10:21.612632 (+ 16169us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.612636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.612647 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:10:21.612680 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.612682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.612699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.612924 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:21.613003 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.614059 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.615318 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.616011 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.616914 (+ 903us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.616976 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.617001 (+ 25us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.617039 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16132,"thread_start_us":55,"threads_started":1}
I20260927 11:10:21.617837 30656 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6372bd453f4cfdb852dad61682d0d6 (DEFAULT_TABLE table=test-table [id=67673d753e16403ebf61f1f255bfef8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:21.617949 31023 tablet_bootstrap.cc:492] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5: Bootstrap starting.
I20260927 11:10:21.618156 30656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6372bd453f4cfdb852dad61682d0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:21.618556 31023 tablet_bootstrap.cc:654] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:21.621701 31025 tablet_bootstrap.cc:492] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: Bootstrap starting.
I20260927 11:10:21.622362 31025 tablet_bootstrap.cc:654] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:21.622710 31025 log.cc:824] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:21.623739 31025 tablet_bootstrap.cc:492] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: No bootstrap required, opened a new log
I20260927 11:10:21.623800 31025 ts_tablet_manager.cc:1397] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:10:21.625988 31025 raft_consensus.cc:348] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.626127 31025 raft_consensus.cc:374] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:21.626153 31025 raft_consensus.cc:729] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6cc229d010a4240a0290988f1fabfe0, State: Initialized, Role: FOLLOWER
I20260927 11:10:21.626261 31025 consensus_queue.cc:260] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.626477 31025 ts_tablet_manager.cc:1428] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:10:21.626768 30722 heartbeater.cc:503] Master 127.17.95.254:33605 was elected leader, sending a full tablet report...
I20260927 11:10:21.618873 31023 log.cc:824] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:21.627420 31024 tablet_bootstrap.cc:492] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Bootstrap starting.
I20260927 11:10:21.628034 31024 tablet_bootstrap.cc:654] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:21.628340 31024 log.cc:824] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:21.628787 31023 tablet_bootstrap.cc:492] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5: No bootstrap required, opened a new log
I20260927 11:10:21.628859 31023 ts_tablet_manager.cc:1397] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260927 11:10:21.631096 31023 raft_consensus.cc:348] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.631242 31023 raft_consensus.cc:374] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:21.631282 31023 raft_consensus.cc:729] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4184c84570404cda94cf5e4dcdcadde5, State: Initialized, Role: FOLLOWER
I20260927 11:10:21.631407 31023 consensus_queue.cc:260] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.631669 31023 ts_tablet_manager.cc:1428] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:10:21.631788 30856 heartbeater.cc:503] Master 127.17.95.254:33605 was elected leader, sending a full tablet report...
I20260927 11:10:21.633306 31024 tablet_bootstrap.cc:492] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: No bootstrap required, opened a new log
I20260927 11:10:21.633386 31024 ts_tablet_manager.cc:1397] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260927 11:10:21.635879 31024 raft_consensus.cc:348] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.636030 31024 raft_consensus.cc:374] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:21.636078 31024 raft_consensus.cc:729] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670ef349b0374d85b8736203531f36b3, State: Initialized, Role: FOLLOWER
I20260927 11:10:21.636206 31024 consensus_queue.cc:260] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.636458 31024 ts_tablet_manager.cc:1428] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:10:21.636569 30990 heartbeater.cc:503] Master 127.17.95.254:33605 was elected leader, sending a full tablet report...
W20260927 11:10:21.675113 30723 tablet.cc:2369] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:10:21.688441 31029 raft_consensus.cc:482] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:21.688581 31029 raft_consensus.cc:504] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.689015 31029 leader_election.cc:288] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6cc229d010a4240a0290988f1fabfe0 (127.17.95.193:45415), 670ef349b0374d85b8736203531f36b3 (127.17.95.195:43765)
I20260927 11:10:21.695699 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.689312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45415 (local address 127.17.95.194:52377)
0927 11:10:21.692455 (+ 3143us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.692466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:21.692661 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:21.692815 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:21.692816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:21.692817 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:21.692877 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.692879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.693722 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.693723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.694819 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.694821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.695477 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.695478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.695545 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.695548 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:21.695586 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 11:10:21.695618 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3121,"mutex_wait_us":35,"thread_start_us":87,"threads_started":1}
I20260927 11:10:21.695802 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.689868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:47525 (local address 127.17.95.195:43765)
0927 11:10:21.689907 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.689909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.692606 (+ 2697us) server_negotiation.cc:407] Connection header received
0927 11:10:21.692661 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.692662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.692673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.692736 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:21.692974 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.693635 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.694938 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.695462 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.695518 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.695524 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.695710 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.695742 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:10:21.695901 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.689519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:52377 (local address 127.17.95.193:45415)
0927 11:10:21.689576 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.689578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.692520 (+ 2942us) server_negotiation.cc:407] Connection header received
0927 11:10:21.692725 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.692726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.692735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.692760 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:21.692937 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.693617 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.694901 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.695354 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.695425 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.695433 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.695821 (+ 388us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.695854 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:10:21.696123 30673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6372bd453f4cfdb852dad61682d0d6" candidate_uuid: "4184c84570404cda94cf5e4dcdcadde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6cc229d010a4240a0290988f1fabfe0" is_pre_election: true
I20260927 11:10:21.696244 30673 raft_consensus.cc:2454] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4184c84570404cda94cf5e4dcdcadde5 in term 0.
I20260927 11:10:21.696429 30745 leader_election.cc:302] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4184c84570404cda94cf5e4dcdcadde5, d6cc229d010a4240a0290988f1fabfe0; no voters:
I20260927 11:10:21.696566 31029 raft_consensus.cc:2781] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:10:21.696615 31029 raft_consensus.cc:482] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:10:21.696636 31029 raft_consensus.cc:3037] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:21.696565 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.689717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:43765 (local address 127.17.95.194:47525)
0927 11:10:21.692555 (+ 2838us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.692563 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:21.692606 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:21.692792 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:21.692793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:21.692795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:21.692901 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.692903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.693721 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.693721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.694849 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:21.694851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.695573 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.695574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:21.695643 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.695646 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:21.696327 (+ 681us) client_negotiation.cc:240] Negotiation successful
0927 11:10:21.696383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2828,"mutex_wait_us":85,"thread_start_us":66,"threads_started":1}
I20260927 11:10:21.697041 30944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6372bd453f4cfdb852dad61682d0d6" candidate_uuid: "4184c84570404cda94cf5e4dcdcadde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670ef349b0374d85b8736203531f36b3" is_pre_election: true
I20260927 11:10:21.697168 30944 raft_consensus.cc:2454] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4184c84570404cda94cf5e4dcdcadde5 in term 0.
I20260927 11:10:21.697456 31029 raft_consensus.cc:504] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.697575 31029 leader_election.cc:288] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [CANDIDATE]: Term 1 election: Requested vote from peers d6cc229d010a4240a0290988f1fabfe0 (127.17.95.193:45415), 670ef349b0374d85b8736203531f36b3 (127.17.95.195:43765)
I20260927 11:10:21.697731 30673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6372bd453f4cfdb852dad61682d0d6" candidate_uuid: "4184c84570404cda94cf5e4dcdcadde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6cc229d010a4240a0290988f1fabfe0"
I20260927 11:10:21.697742 30944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6372bd453f4cfdb852dad61682d0d6" candidate_uuid: "4184c84570404cda94cf5e4dcdcadde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670ef349b0374d85b8736203531f36b3"
I20260927 11:10:21.697795 30673 raft_consensus.cc:3037] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:21.697808 30944 raft_consensus.cc:3037] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:21.698570 30673 raft_consensus.cc:2454] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4184c84570404cda94cf5e4dcdcadde5 in term 1.
I20260927 11:10:21.698632 30944 raft_consensus.cc:2454] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4184c84570404cda94cf5e4dcdcadde5 in term 1.
I20260927 11:10:21.698712 30745 leader_election.cc:302] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4184c84570404cda94cf5e4dcdcadde5, d6cc229d010a4240a0290988f1fabfe0; no voters:
I20260927 11:10:21.698782 31029 raft_consensus.cc:2781] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:21.698887 31029 raft_consensus.cc:686] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [term 1 LEADER]: Becoming Leader. State: Replica: 4184c84570404cda94cf5e4dcdcadde5, State: Running, Role: LEADER
I20260927 11:10:21.698972 31029 consensus_queue.cc:237] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } }
I20260927 11:10:21.699709 30524 catalog_manager.cc:5495] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4184c84570404cda94cf5e4dcdcadde5 (127.17.95.194). New cstate: current_term: 1 leader_uuid: "4184c84570404cda94cf5e4dcdcadde5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4184c84570404cda94cf5e4dcdcadde5" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 } health_report { overall_health: UNKNOWN } } }
I20260927 11:10:21.725816 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.711452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.17.95.194:43329)
0927 11:10:21.711485 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:21.711487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:21.711493 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:10:21.711520 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:21.711520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:21.711536 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:21.711590 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:10:21.711756 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.712439 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.724691 (+ 12252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:21.724984 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:21.725061 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:21.725272 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:10:21.725397 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:10:21.725471 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:21.725697 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 11:10:21.725722 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:10:21.727766 30944 raft_consensus.cc:1261] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 1 FOLLOWER]: Refusing update from remote peer 4184c84570404cda94cf5e4dcdcadde5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:21.728086 31029 consensus_queue.cc:1035] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "670ef349b0374d85b8736203531f36b3" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Sep 27 11:10:21 dist-test-slave-6j0v krb5kdc[29528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for HTTP/127.17.95.212@KRBTEST.COM
I20260927 11:10:21.730836 30673 raft_consensus.cc:1261] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 1 FOLLOWER]: Refusing update from remote peer 4184c84570404cda94cf5e4dcdcadde5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:21.731098 31029 consensus_queue.cc:1035] T 3d6372bd453f4cfdb852dad61682d0d6 P 4184c84570404cda94cf5e4dcdcadde5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6cc229d010a4240a0290988f1fabfe0" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:21.734366 31038 mvcc.cc:205] Tried to move back new op lower bound from 7333918399393394688 to 7333918399279611904. Current Snapshot: MvccSnapshot[applied={T|T < 7333918399393394688}]
I20260927 11:10:21.734853 31041 mvcc.cc:205] Tried to move back new op lower bound from 7333918399393394688 to 7333918399279611904. Current Snapshot: MvccSnapshot[applied={T|T < 7333918399393394688}]
I20260927 11:10:21.736964 31039 mvcc.cc:205] Tried to move back new op lower bound from 7333918399393394688 to 7333918399279611904. Current Snapshot: MvccSnapshot[applied={T|T < 7333918399393394688}]
I20260927 11:10:21.739141 30524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:21.739224 30524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:21.739861 30524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f835cc839fd4cbbab7035b9d6802f09: Sending DeleteTablet for 3 replicas of tablet 3d6372bd453f4cfdb852dad61682d0d6
I20260927 11:10:21.740245 30773 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6372bd453f4cfdb852dad61682d0d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:21 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:41412
I20260927 11:10:21.740255 30911 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6372bd453f4cfdb852dad61682d0d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:21 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:57062
I20260927 11:10:21.740556 30656 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6372bd453f4cfdb852dad61682d0d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:21 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:36630
I20260927 11:10:21.740767 31049 tablet_replica.cc:330] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0: stopping tablet replica
I20260927 11:10:21.740818 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30562
I20260927 11:10:21.740916 31049 raft_consensus.cc:2229] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:21.741056 31049 raft_consensus.cc:2258] T 3d6372bd453f4cfdb852dad61682d0d6 P d6cc229d010a4240a0290988f1fabfe0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 11:10:21.748456 30510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:45415 (error 108)
I20260927 11:10:21.749014 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30726
I20260927 11:10:21.755841 31048 tablet_replica.cc:330] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: stopping tablet replica
I20260927 11:10:21.755985 31048 raft_consensus.cc:2229] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:21.756114 31048 raft_consensus.cc:2258] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:21.759604 31020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.759463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45415 (local address 127.0.0.1:36636)
0927 11:10:21.759514 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.759549 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 11:10:21.759795 30510 catalog_manager.cc:4584] TS d6cc229d010a4240a0290988f1fabfe0 (127.17.95.193:45415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d6372bd453f4cfdb852dad61682d0d6: Network error: Client connection negotiation failed: client connection to 127.17.95.193:45415: connect: Connection refused (error 111)
I20260927 11:10:21.763557 31048 ts_tablet_manager.cc:1905] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:21.765722 31048 ts_tablet_manager.cc:1918] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:21.765849 31048 log.cc:1196] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/3d6372bd453f4cfdb852dad61682d0d6
I20260927 11:10:21.765900 31020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:21.765806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:43329 (local address 127.0.0.1:41436)
0927 11:10:21.765847 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:21.765868 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.194:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 11:10:21.766268 31048 ts_tablet_manager.cc:1939] T 3d6372bd453f4cfdb852dad61682d0d6 P 670ef349b0374d85b8736203531f36b3: Deleting consensus metadata
I20260927 11:10:21.766728 30511 catalog_manager.cc:4841] TS 670ef349b0374d85b8736203531f36b3 (127.17.95.195:43765): tablet 3d6372bd453f4cfdb852dad61682d0d6 (table test-table [id=67673d753e16403ebf61f1f255bfef8b]) successfully deleted
I20260927 11:10:21.767014 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30859
I20260927 11:10:21.771979 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 30491
I20260927 11:10:21.779441 17791 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 11:10:22.124131 17791 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 11:10:22.124356 17791 mini_ranger.cc:66] Stopping Ranger...
I20260927 11:10:22.468425 17791 mini_ranger.cc:68] Stopped Ranger
2026-09-27 11:10:22.468 UTC [29558] LOG: received smart shutdown request
2026-09-27 11:10:22.471 UTC [29558] LOG: background worker "logical replication launcher" (PID 29567) exited with exit code 1
2026-09-27 11:10:22.472 UTC [29562] LOG: shutting down
2026-09-27 11:10:22.623 UTC [29558] LOG: database system is shut down
2026-09-27T11:10:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45163 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:10:22 dist-test-slave-6j0v krb5kdc[31060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:10:22 dist-test-slave-6j0v krb5kdc[31060](info): set up 2 sockets
Sep 27 11:10:22 dist-test-slave-6j0v krb5kdc[31060](info): commencing operation
krb5kdc: starting...
W20260927 11:10:23.766891 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:10:23 dist-test-slave-6j0v krb5kdc[31060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:10:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:10:23.878007 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34037
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34037,127.17.95.253:38161,127.17.95.252:46813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:23.955917 31073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:23.956107 31073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:23.956130 31073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:23.957196 31073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:23.957231 31073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:23.957245 31073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:23.957257 31073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:23.957269 31073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:23.958679 31073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34037,127.17.95.253:38161,127.17.95.252:46813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:34037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:23.958859 31073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:23.959018 31073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:23.963792 31079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:23.963856 31081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:23.963804 31078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:23.964184 31073 server_base.cc:1034] running on GCE node
I20260927 11:10:23.964448 31073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:23.964761 31073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:23.965924 31073 hybrid_clock.cc:648] HybridClock initialized: now 1790507423965902 us; error 40 us; skew 500 ppm
Sep 27 11:10:23 dist-test-slave-6j0v krb5kdc[31060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:23.968362 31073 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:23.968729 31073 webserver.cc:460] Webserver started at http://127.17.95.254:46647/ using document root <none> and password file <none>
I20260927 11:10:23.968891 31073 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:23.968928 31073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:23.969009 31073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:23.969966 31073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "4fe5f8ba0577486c9c83d53cafb2b511"
format_stamp: "Formatted at 2026-09-27 11:10:23 on dist-test-slave-6j0v"
server_key: "c74ca7c6fbd6e9f4baf30b05ea716d22"
server_key_iv: "e5f2b345330c59d1dd96aa0a95756a7f"
server_key_version: "encryptionkey@0"
I20260927 11:10:23.970283 31073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "4fe5f8ba0577486c9c83d53cafb2b511"
format_stamp: "Formatted at 2026-09-27 11:10:23 on dist-test-slave-6j0v"
server_key: "c74ca7c6fbd6e9f4baf30b05ea716d22"
server_key_iv: "e5f2b345330c59d1dd96aa0a95756a7f"
server_key_version: "encryptionkey@0"
I20260927 11:10:23.971606 31073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:10:23.972421 31088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:23.972667 31073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:23.972771 31073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "4fe5f8ba0577486c9c83d53cafb2b511"
format_stamp: "Formatted at 2026-09-27 11:10:23 on dist-test-slave-6j0v"
server_key: "c74ca7c6fbd6e9f4baf30b05ea716d22"
server_key_iv: "e5f2b345330c59d1dd96aa0a95756a7f"
server_key_version: "encryptionkey@0"
I20260927 11:10:23.972846 31073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:23.988179 31073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:23.990403 31073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:23.990567 31073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:23.994520 31073 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:34037
I20260927 11:10:23.994619 31140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:34037 every 8 connection(s)
I20260927 11:10:23.995119 31073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:23.995805 31141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:23.997334 31141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:10:24.002287 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31073
I20260927 11:10:24.002418 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:10:24.002586 17791 external_mini_cluster.cc:1428] Setting key ed668decd1fcc3de90d9212fc05b4708
Sep 27 11:10:23 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:24.006946 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:23.997975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34037 (local address 127.0.0.1:54512)
0927 11:10:23.998135 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:23.998168 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:23.998653 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:23.998943 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:23.998946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:23.999234 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:23.999376 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:23.999380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.000364 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.000365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.001024 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.001026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.001522 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.001523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.001598 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.002032 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.002040 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.003672 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.005458 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.005464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.005466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.005662 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.005917 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.005918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.005919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.006006 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.006362 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.006366 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.006428 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.006534 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.006777 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":106,"thread_start_us":89,"threads_started":1}
I20260927 11:10:24.006946 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:23.998145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54512 (local address 127.17.95.254:34037)
0927 11:10:23.998264 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:23.998268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:23.998300 (+ 32us) server_negotiation.cc:407] Connection header received
0927 11:10:23.998818 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:23.998820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:23.998845 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:23.998936 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:23.999475 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.000267 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.001111 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.001427 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.001550 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.003769 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.003776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.003776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.003784 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.005353 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.005760 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.005763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.005763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.005847 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.006071 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.006072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.006072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.006291 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.006380 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.006579 (+ 199us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.006714 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":41,"threads_started":1}
I20260927 11:10:24.007736 31141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:10:24.008062 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.007971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32840)
0927 11:10:24.008003 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.008042 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260927 11:10:24.008225 31091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
I20260927 11:10:24.008656 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.008565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32850)
0927 11:10:24.008614 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.008639 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 11:10:24.008776 31141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:38161: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.95.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab.
Entry for principal kudu/127.17.95.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab.
I20260927 11:10:24.026407 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:38161
--webserver_interface=127.17.95.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab
--principal=kudu/127.17.95.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34037,127.17.95.253:38161,127.17.95.252:46813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:10:24.058000 31141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } attempt: 1
I20260927 11:10:24.058452 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.058325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32854)
0927 11:10:24.058385 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.058423 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 11:10:24.058801 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.058708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32868)
0927 11:10:24.058753 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.058776 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 11:10:24.058885 31141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:38161: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
W20260927 11:10:24.106236 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:24.106462 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:24.106503 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:24.107949 31148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:24.108032 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:24.108062 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:24.108120 31148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:24.108147 31148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:24.109826 31148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34037,127.17.95.253:38161,127.17.95.252:46813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab
--principal=kudu/127.17.95.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.95.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:24.110172 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:24.110411 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:24.115172 31154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:24.115154 31153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:24.115154 31156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:24.115463 31148 server_base.cc:1034] running on GCE node
I20260927 11:10:24.115758 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:24.116096 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:24.117292 31148 hybrid_clock.cc:648] HybridClock initialized: now 1790507424117274 us; error 27 us; skew 500 ppm
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:24.119756 31148 init.cc:377] Logged in from keytab as kudu/127.17.95.253@KRBTEST.COM (short username kudu)
I20260927 11:10:24.120242 31148 webserver.cc:460] Webserver started at http://127.17.95.253:39997/ using document root <none> and password file <none>
I20260927 11:10:24.120479 31148 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:24.120549 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:24.120703 31148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:24.121784 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data/instance:
uuid: "019055031db849e6b5a2c7476c3bdd18"
format_stamp: "Formatted at 2026-09-27 11:10:24 on dist-test-slave-6j0v"
server_key: "00aaeb1a2cd164d292303e7232b7fbb3"
server_key_iv: "155eae99d35e29fe4454d1d6b5bff749"
server_key_version: "encryptionkey@0"
I20260927 11:10:24.122150 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal/instance:
uuid: "019055031db849e6b5a2c7476c3bdd18"
format_stamp: "Formatted at 2026-09-27 11:10:24 on dist-test-slave-6j0v"
server_key: "00aaeb1a2cd164d292303e7232b7fbb3"
server_key_iv: "155eae99d35e29fe4454d1d6b5bff749"
server_key_version: "encryptionkey@0"
I20260927 11:10:24.123620 31148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:10:24.124320 31163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:24.124487 31148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:10:24.124590 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal
uuid: "019055031db849e6b5a2c7476c3bdd18"
format_stamp: "Formatted at 2026-09-27 11:10:24 on dist-test-slave-6j0v"
server_key: "00aaeb1a2cd164d292303e7232b7fbb3"
server_key_iv: "155eae99d35e29fe4454d1d6b5bff749"
server_key_version: "encryptionkey@0"
I20260927 11:10:24.124676 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:24.127029 31141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } attempt: 2
I20260927 11:10:24.127475 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.127346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32884)
0927 11:10:24.127404 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.127435 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 11:10:24.127827 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.127736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32886)
0927 11:10:24.127781 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.127804 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 11:10:24.127918 31141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:38161: Network error: Client connection negotiation failed: client connection to 127.17.95.253:38161: connect: Connection refused (error 111)
I20260927 11:10:24.153015 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:24.154976 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:24.155174 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:24.159489 31148 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.253:38161
I20260927 11:10:24.159562 31215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.253:38161 every 8 connection(s)
I20260927 11:10:24.159898 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
I20260927 11:10:24.160825 31216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:24.160969 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31148
I20260927 11:10:24.161094 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-1/wal/instance
I20260927 11:10:24.161283 17791 external_mini_cluster.cc:1428] Setting key 2a80c13006fb4ef8b81a1458189dd199
I20260927 11:10:24.163142 31216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.95.252@KRBTEST.COM; defaulting to no policy
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.252@KRBTEST.COM" created.
I20260927 11:10:24.172770 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.163855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.17.95.254:34037)
0927 11:10:24.163927 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.163929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.164126 (+ 197us) server_negotiation.cc:407] Connection header received
0927 11:10:24.164544 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.164546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.164563 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.164645 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.165322 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.166161 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.167209 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.167560 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.167650 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.169732 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.169733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.169734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.169740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.171341 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.171820 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.171821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.171822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.171881 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.172137 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.172137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.172138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.172248 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.172321 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.172597 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.172677 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:10:24.172960 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.163788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34037 (local address 127.0.0.1:54516)
0927 11:10:24.164015 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.164050 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.164452 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.164709 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.164712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.165021 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.165219 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.165223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.166278 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.166279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.167117 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.167120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.167651 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.167651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.167752 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.168235 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.168243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.169599 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.171441 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.171448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.171450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.171744 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.171947 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.171947 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.171948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.172061 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.172313 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.172320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.172455 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.172575 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.172721 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":71,"threads_started":1}
I20260927 11:10:24.173425 31216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:10:24.180745 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.173676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32892)
0927 11:10:24.173722 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.173732 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.173767 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.174028 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.174028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.174172 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.174211 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.174213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.175099 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.175100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.175669 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.175670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.176166 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.176167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.176231 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.176676 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.176681 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.177781 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.179645 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.179646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.179646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.179820 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.180073 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.180074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.180075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.180128 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.180411 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.180412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.180448 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.180520 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.180638 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 11:10:24.180815 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.173758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32892 (local address 127.17.95.253:38161)
0927 11:10:24.173905 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.173909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.173924 (+ 15us) server_negotiation.cc:407] Connection header received
0927 11:10:24.173940 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.173941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.173951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.174028 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.174284 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.174945 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.175759 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.176064 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.176144 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.177862 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.177870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.177870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.177877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.179556 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.179943 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.179946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.179946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.180001 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.180197 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.180198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.180199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.180342 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.180433 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.180558 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.180688 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260927 11:10:24.181473 31216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:10:24.181825 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.181712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:51974)
0927 11:10:24.181759 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.181807 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 11:10:24.181959 31164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
I20260927 11:10:24.182365 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.182277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:51988)
0927 11:10:24.182325 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.182349 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 11:10:24.182475 31216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:46813: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab.
Entry for principal kudu/127.17.95.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab.
I20260927 11:10:24.185454 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:46813
--webserver_interface=127.17.95.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab
--principal=kudu/127.17.95.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:34037,127.17.95.253:38161,127.17.95.252:46813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:10:24.219137 31141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } attempt: 3
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:10:24.226336 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.219565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.17.95.253:38161)
0927 11:10:24.219634 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.219636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.219649 (+ 13us) server_negotiation.cc:407] Connection header received
0927 11:10:24.219711 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.219712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.219724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.219796 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.220126 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.220598 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.221310 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.221543 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.221631 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.223679 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.223680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.223681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.223688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.225255 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.225622 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.225623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.225623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.225678 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.225888 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.225889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.225889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.225993 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.226056 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.226209 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.226271 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 11:10:24.226370 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.219468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32908)
0927 11:10:24.219529 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.219548 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.219641 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.219799 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.219800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.219956 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.220020 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.220022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.220679 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.220680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.221219 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.221221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.221632 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.221633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.221721 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.222170 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.222183 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.223574 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.225344 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.225345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.225345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.225549 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.225747 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.225748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.225748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.225821 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.226057 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.226058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.226107 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.226175 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.226275 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 11:10:24.226837 31141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:10:24.227208 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.227050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:51996)
0927 11:10:24.227125 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.227184 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 11:10:24.227530 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.227451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52012)
0927 11:10:24.227475 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.227497 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260927 11:10:24.227613 31141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:46813: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
I20260927 11:10:24.231719 31216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } attempt: 1
I20260927 11:10:24.232075 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.231957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52016)
0927 11:10:24.232012 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.232041 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 11:10:24.232393 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.232292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52022)
0927 11:10:24.232348 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.232372 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 11:10:24.232486 31216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:46813: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
I20260927 11:10:24.258780 31141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } attempt: 1
I20260927 11:10:24.259223 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.259105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52038)
0927 11:10:24.259168 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.259195 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 11:10:24.259616 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.259537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52040)
0927 11:10:24.259579 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.259602 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 11:10:24.259721 31141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:46813: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
W20260927 11:10:24.264938 31223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:24.265103 31223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:24.265123 31223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:24.266196 31223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:24.266232 31223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:24.266244 31223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:24.266294 31223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:24.266311 31223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:24.267840 31223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:34037,127.17.95.253:38161,127.17.95.252:46813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab
--principal=kudu/127.17.95.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:46813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.95.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:24.268069 31223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:24.268286 31223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:24.272746 31231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:24.272754 31229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:24.272747 31228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:24.272977 31223 server_base.cc:1034] running on GCE node
I20260927 11:10:24.273288 31223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:24.273573 31223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:24.274729 31223 hybrid_clock.cc:648] HybridClock initialized: now 1790507424274724 us; error 35 us; skew 500 ppm
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:24.277297 31223 init.cc:377] Logged in from keytab as kudu/127.17.95.252@KRBTEST.COM (short username kudu)
I20260927 11:10:24.277709 31223 webserver.cc:460] Webserver started at http://127.17.95.252:43173/ using document root <none> and password file <none>
I20260927 11:10:24.277894 31223 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:24.277935 31223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:24.278015 31223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:24.278954 31223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data/instance:
uuid: "75f01597177646fea6c1337198bd21c9"
format_stamp: "Formatted at 2026-09-27 11:10:24 on dist-test-slave-6j0v"
server_key: "cd1bd12d1a6eabde64949413c37d6270"
server_key_iv: "98a819ef801a6a56d5f83456ba58573f"
server_key_version: "encryptionkey@0"
I20260927 11:10:24.279316 31223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal/instance:
uuid: "75f01597177646fea6c1337198bd21c9"
format_stamp: "Formatted at 2026-09-27 11:10:24 on dist-test-slave-6j0v"
server_key: "cd1bd12d1a6eabde64949413c37d6270"
server_key_iv: "98a819ef801a6a56d5f83456ba58573f"
server_key_version: "encryptionkey@0"
I20260927 11:10:24.280509 31223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:24.281175 31238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:24.281381 31223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:24.281466 31223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal
uuid: "75f01597177646fea6c1337198bd21c9"
format_stamp: "Formatted at 2026-09-27 11:10:24 on dist-test-slave-6j0v"
server_key: "cd1bd12d1a6eabde64949413c37d6270"
server_key_iv: "98a819ef801a6a56d5f83456ba58573f"
server_key_version: "encryptionkey@0"
I20260927 11:10:24.281522 31223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:24.299572 31223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:24.300647 31216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } attempt: 2
I20260927 11:10:24.301105 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.300987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52054)
0927 11:10:24.301037 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.301074 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 11:10:24.301472 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.301371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52062)
0927 11:10:24.301416 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.301446 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 11:10:24.301609 31216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:46813: Network error: Client connection negotiation failed: client connection to 127.17.95.252:46813: connect: Connection refused (error 111)
I20260927 11:10:24.302160 31223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:24.302320 31223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:24.306093 31223 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.252:46813
I20260927 11:10:24.306157 31290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.252:46813 every 8 connection(s)
I20260927 11:10:24.306494 31223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
I20260927 11:10:24.307421 31291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:24.308964 31291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:10:24.309870 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31223
I20260927 11:10:24.309999 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790507277271108-17791-0/minicluster-data/master-2/wal/instance
I20260927 11:10:24.310232 17791 external_mini_cluster.cc:1428] Setting key e731fb07304481f44ebebe39e957485a
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:10:24.319633 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.309752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.17.95.254:34037)
0927 11:10:24.309829 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.309831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.310049 (+ 218us) server_negotiation.cc:407] Connection header received
0927 11:10:24.310610 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.310611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.310622 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.310691 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.311446 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.312012 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.313022 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.313217 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.313300 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.315694 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.315695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.315696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.315703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.318152 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.318652 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.318653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.318653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.318726 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.318997 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.318998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.318998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.319168 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.319236 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.319446 (+ 210us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.319528 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:10:24.319860 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.310626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.17.95.252:46813)
0927 11:10:24.310792 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.310797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.310920 (+ 123us) server_negotiation.cc:407] Connection header received
0927 11:10:24.310946 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.310948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.310971 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.311057 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.311742 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.312556 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.313332 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.313665 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.313744 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.316277 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.316284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.316284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.316291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.318122 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.318655 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.318659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.318659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.318764 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.318997 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.318998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.318998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.319294 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.319445 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.319540 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.319647 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260927 11:10:24.319869 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.309679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:34037 (local address 127.0.0.1:54532)
0927 11:10:24.309899 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.309943 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.310498 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.310766 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.310770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.311173 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.311348 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.311353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.312110 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.312111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.312932 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.312935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.313308 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.313309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.313472 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.314000 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.314009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.315606 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.318234 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.318242 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.318244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.318554 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.318811 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.318811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.318812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.318926 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.319236 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.319241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.319298 (+ 57us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.319416 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.319593 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":92,"thread_start_us":98,"threads_started":1}
I20260927 11:10:24.320549 31291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:24.327767 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.320799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:38161 (local address 127.0.0.1:32910)
0927 11:10:24.320864 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.320873 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.320908 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.321065 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.321066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.321213 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.321269 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.321271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.321898 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.321899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.322383 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.322384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.322780 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.322781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.322859 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.323316 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.323321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.324557 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.326512 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.326513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.326513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.326702 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.326927 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.326928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.326928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.327059 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.327433 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.327434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.327479 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.327572 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.327667 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 11:10:24.327768 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.320887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.17.95.253:38161)
0927 11:10:24.320933 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.320935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.320946 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:10:24.320974 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.320975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.320986 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.321051 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.321342 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.321794 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.322476 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.322691 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.322780 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.324672 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.324673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.324673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.324680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.326409 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.326785 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.326787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.326787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.326850 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.327190 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.327191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.327192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.327357 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.327442 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.327600 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.327675 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:10:24.328205 31291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:10:24.328727 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.320958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54544 (local address 127.17.95.254:34037)
0927 11:10:24.320995 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.320997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.321003 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:10:24.321023 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.321023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.321034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.321089 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.321634 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.322062 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.322722 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.322919 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.323002 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.325359 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.325361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.325362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.325370 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.327017 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.327437 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.327438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.327439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.327495 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.327778 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.327779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.327779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.328060 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.328240 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.328433 (+ 193us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.328518 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:10:24.334861 31141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } attempt: 2
I20260927 11:10:24.336097 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.328474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52092)
0927 11:10:24.328504 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.328515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.328560 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.328759 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.328760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.328871 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.328922 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.328924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.329901 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.329902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.330527 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.330530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.331041 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.331042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.331136 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.331492 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.331497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.332796 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.334568 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.334569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.334569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.334777 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.335012 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.335013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.335013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.335110 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.335804 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.335805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.335852 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.335917 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.336014 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260927 11:10:24.336725 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.328508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.17.95.252:46813)
0927 11:10:24.328646 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.328648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.328657 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:10:24.328678 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.328679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.328690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.328749 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.329002 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.329826 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.330608 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.330951 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.331019 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.332882 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.332883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.332883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.332889 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.334485 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.334857 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.334859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.334859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.334918 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.335170 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.335171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.335172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.335694 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.335812 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.336603 (+ 791us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.336668 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260927 11:10:24.337675 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.329524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.17.95.253:38161)
0927 11:10:24.329589 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.329591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.329615 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:10:24.329635 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.329636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.329646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.329702 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.330247 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.330845 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.331805 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.331995 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.332104 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.334740 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.334747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.334748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.334754 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.336488 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.336858 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.336859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.336859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.336924 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.337154 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.337155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.337155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.337279 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.337470 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.337506 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.337587 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 11:10:24.338689 31291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: Bootstrap starting.
I20260927 11:10:24.339567 31291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:24.339620 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:10:24.339874 31291 log.cc:824] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: Log is configured to *not* fsync() on all Append() calls
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507423, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:10:24.340664 31291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: No bootstrap required, opened a new log
I20260927 11:10:24.342705 31291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.342849 31291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:24.342882 31291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f01597177646fea6c1337198bd21c9, State: Initialized, Role: FOLLOWER
I20260927 11:10:24.343194 31291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.343632 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.335243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52104)
0927 11:10:24.335294 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.335304 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.335344 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.335596 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.335597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.335735 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.335796 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.335797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.337066 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.337067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.337828 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.337832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.338418 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.338419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.338473 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.338890 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.338894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.340304 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.342398 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.342400 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.342400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.342577 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.342802 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.342803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.342804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.342862 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.343301 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.343302 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.343348 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.343417 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.343514 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 11:10:24.343631 31291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:24.344259 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.335279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.17.95.252:46813)
0927 11:10:24.335461 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.335462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.335468 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:10:24.335483 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.335483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.335490 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.335588 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.335889 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.336950 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.338024 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.338311 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.338633 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.340408 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.340409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.340410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.340416 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.342306 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.342652 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.342653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.342653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.342730 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.342929 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.342930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.342930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.343202 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.343270 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.344050 (+ 780us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.344144 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260927 11:10:24.346045 31141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: Bootstrap starting.
W20260927 11:10:24.345943 31316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260927 11:10:24.347996 31141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:24.348662 31141 log.cc:824] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:24.347059 31223 master_runner.cc:186] Error getting master registration for 127.17.95.254:34037: OK, instance_id { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" instance_seqno: 1790507423993382 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 11:10:24.349589 31223 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 11:10:24.349752 31141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: No bootstrap required, opened a new log
I20260927 11:10:24.350831 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.341521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.17.95.253:38161)
0927 11:10:24.341559 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.341560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.342760 (+ 1200us) server_negotiation.cc:407] Connection header received
0927 11:10:24.342855 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.342855 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.342865 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.342896 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.343909 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.344632 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.345568 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.345836 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.345882 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.347431 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.347432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.347432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.347439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.349064 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.349450 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.349451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.349452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.349504 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.349774 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.349775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.349775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.349899 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.350415 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.350657 (+ 242us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.350734 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 11:10:24.347085 31316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:10:24.351289 31305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.351475 31305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:10:24.351805 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.341139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54558 (local address 127.17.95.254:34037)
0927 11:10:24.341190 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.341192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.341588 (+ 396us) server_negotiation.cc:407] Connection header received
0927 11:10:24.341695 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.341697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.341708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.341778 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.342432 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.343105 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.344175 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.344490 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.344532 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.346259 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.346260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.346260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.346267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.350331 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.350851 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.350852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.350852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.350909 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.351172 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.351173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.351173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.351309 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.351401 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.351639 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.351720 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 11:10:24.352438 31141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.352604 31141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:24.352685 31141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fe5f8ba0577486c9c83d53cafb2b511, State: Initialized, Role: FOLLOWER
I20260927 11:10:24.352859 31141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.353195 31141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 11:10:24.355167 31330 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:10:24.355258 31330 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:10:24.353183 31319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.355525 31319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:10:24.358117 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.341392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.17.95.252:46813)
0927 11:10:24.341429 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.341432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.351290 (+ 9858us) server_negotiation.cc:407] Connection header received
0927 11:10:24.351405 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.351406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.351416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.351444 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.351994 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.352619 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.353509 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.353736 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.353791 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.355360 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.355361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.355362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.355368 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.356902 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.357302 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.357303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.357304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.357358 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.357555 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.357555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.357556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.357682 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.357746 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.357961 (+ 215us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.358035 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:10:24.355783 31073 master_runner.cc:186] Error getting master registration for 127.17.95.253:38161: OK, instance_id { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" instance_seqno: 1790507424158254 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 11:10:24.358685 31073 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 11:10:24.392809 31216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } attempt: 3
Sep 27 11:10:24 dist-test-slave-6j0v krb5kdc[31060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:10:24.401249 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.393117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:46813 (local address 127.0.0.1:52124)
0927 11:10:24.393173 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:24.393185 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:24.393224 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:24.393397 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:24.393398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:24.393601 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:24.393655 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.393657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.394568 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.394577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.395419 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:24.395421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.395884 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.395886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:24.395946 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.396510 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:24.396516 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:24.398090 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:24.400104 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.400105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.400106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.400322 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.400547 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:24.400548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:24.400548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:24.400645 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:24.400925 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:24.400926 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:24.400974 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 11:10:24.401046 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 11:10:24.401142 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 11:10:24.401261 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:24.393194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.17.95.252:46813)
0927 11:10:24.393246 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:24.393248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:24.393279 (+ 31us) server_negotiation.cc:407] Connection header received
0927 11:10:24.393303 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:24.393304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:24.393315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:24.393376 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:24.393739 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.394473 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.395505 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:24.395798 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:24.395879 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:24.398182 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:24.398184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:24.398184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:24.398192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:24.400006 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.400402 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.400403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.400403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.400469 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:24.400715 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:24.400716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:24.400716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:24.400854 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:24.400930 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:24.401095 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 11:10:24.401173 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 11:10:24.403239 31216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: Bootstrap starting.
I20260927 11:10:24.403980 31216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:24.404326 31216 log.cc:824] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:24.405218 31216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: No bootstrap required, opened a new log
I20260927 11:10:24.407397 31216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.407545 31216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:24.407584 31216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019055031db849e6b5a2c7476c3bdd18, State: Initialized, Role: FOLLOWER
I20260927 11:10:24.407768 31216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.408098 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.408157 31216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:24.408236 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 11:10:24.410202 31344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:10:24.410310 31344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:10:24.412683 31148 master_runner.cc:263] No leader master found from master 019055031db849e6b5a2c7476c3bdd18
I20260927 11:10:24.412734 31148 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 11:10:24.491314 31094 tablet.cc:2369] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:10:24.552954 31244 tablet.cc:2369] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:10:24.615727 31319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:24.615845 31319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.616205 31319 leader_election.cc:288] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019055031db849e6b5a2c7476c3bdd18 (127.17.95.253:38161), 75f01597177646fea6c1337198bd21c9 (127.17.95.252:46813)
I20260927 11:10:24.616452 31265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f01597177646fea6c1337198bd21c9" is_pre_election: true
I20260927 11:10:24.616475 31190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019055031db849e6b5a2c7476c3bdd18" is_pre_election: true
I20260927 11:10:24.616636 31190 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fe5f8ba0577486c9c83d53cafb2b511 in term 0.
I20260927 11:10:24.616637 31265 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fe5f8ba0577486c9c83d53cafb2b511 in term 0.
I20260927 11:10:24.616811 31091 leader_election.cc:302] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019055031db849e6b5a2c7476c3bdd18, 4fe5f8ba0577486c9c83d53cafb2b511; no voters:
I20260927 11:10:24.616961 31319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:10:24.617053 31319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:10:24.617085 31319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:24.617794 31319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.617930 31319 leader_election.cc:288] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [CANDIDATE]: Term 1 election: Requested vote from peers 019055031db849e6b5a2c7476c3bdd18 (127.17.95.253:38161), 75f01597177646fea6c1337198bd21c9 (127.17.95.252:46813)
I20260927 11:10:24.618099 31190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019055031db849e6b5a2c7476c3bdd18"
I20260927 11:10:24.618109 31265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f01597177646fea6c1337198bd21c9"
I20260927 11:10:24.618180 31190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:24.618180 31265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:24.619056 31190 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fe5f8ba0577486c9c83d53cafb2b511 in term 1.
I20260927 11:10:24.619096 31265 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fe5f8ba0577486c9c83d53cafb2b511 in term 1.
I20260927 11:10:24.619252 31091 leader_election.cc:302] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019055031db849e6b5a2c7476c3bdd18, 4fe5f8ba0577486c9c83d53cafb2b511; no voters:
I20260927 11:10:24.619369 31319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:24.619475 31319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [term 1 LEADER]: Becoming Leader. State: Replica: 4fe5f8ba0577486c9c83d53cafb2b511, State: Running, Role: LEADER
I20260927 11:10:24.619606 31319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } }
I20260927 11:10:24.619989 31346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fe5f8ba0577486c9c83d53cafb2b511. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.620080 31346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:24.620311 31348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:24.620488 31348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:24.621846 31190 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [term 1 FOLLOWER]: Refusing update from remote peer 4fe5f8ba0577486c9c83d53cafb2b511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:24.621855 31265 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [term 1 FOLLOWER]: Refusing update from remote peer 4fe5f8ba0577486c9c83d53cafb2b511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:24.622164 31346 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:24.622265 31319 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:24.623428 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fe5f8ba0577486c9c83d53cafb2b511. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.623521 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:10:24.623584 31305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fe5f8ba0577486c9c83d53cafb2b511. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.623677 31305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:10:24.624261 31346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.624344 31346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:24.624481 31305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.624519 31346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.624568 31305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:10:24.624575 31346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:24.624732 31348 catalog_manager.cc:1347] Generated new cluster ID: 8717c5ecdcca4efb9dcca0b1d46f8f2d
I20260927 11:10:24.624727 31350 mvcc.cc:205] Tried to move back new op lower bound from 7333918411248713728 to 7333918411242549248. Current Snapshot: MvccSnapshot[applied={T|T < 7333918411248713728}]
I20260927 11:10:24.624790 31348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:10:24.624835 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5f8ba0577486c9c83d53cafb2b511" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 34037 } } peers { permanent_uuid: "019055031db849e6b5a2c7476c3bdd18" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 38161 } } peers { permanent_uuid: "75f01597177646fea6c1337198bd21c9" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 46813 } } }
I20260927 11:10:24.624922 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:10:24.624981 31351 mvcc.cc:205] Tried to move back new op lower bound from 7333918411248713728 to 7333918411242549248. Current Snapshot: MvccSnapshot[applied={T|T < 7333918411248713728}]
I20260927 11:10:24.651926 31348 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:10:24.652661 31348 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:24.658121 31348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fe5f8ba0577486c9c83d53cafb2b511: Generated new TSK 0
I20260927 11:10:24.658303 31348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:10:25.351578 31316 catalog_manager.cc:1259] Loaded cluster ID: 8717c5ecdcca4efb9dcca0b1d46f8f2d
I20260927 11:10:25.351648 31316 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: loading cluster ID for follower catalog manager: success
I20260927 11:10:25.352375 31316 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: acquiring CA information for follower catalog manager: success
I20260927 11:10:25.352627 31316 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 75f01597177646fea6c1337198bd21c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:10:25.410732 31344 catalog_manager.cc:1259] Loaded cluster ID: 8717c5ecdcca4efb9dcca0b1d46f8f2d
I20260927 11:10:25.410799 31344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: loading cluster ID for follower catalog manager: success
I20260927 11:10:25.411597 31344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: acquiring CA information for follower catalog manager: success
I20260927 11:10:25.411851 31344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 019055031db849e6b5a2c7476c3bdd18: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:10:25.725370 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31073
I20260927 11:10:25.730783 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31148
I20260927 11:10:25.735388 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31223
2026-09-27T11:10:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3006 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:10:25 dist-test-slave-6j0v krb5kdc[31363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:10:25 dist-test-slave-6j0v krb5kdc[31363](info): set up 2 sockets
Sep 27 11:10:25 dist-test-slave-6j0v krb5kdc[31363](info): commencing operation
krb5kdc: starting...
W20260927 11:10:26.743032 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:10:26 dist-test-slave-6j0v krb5kdc[31363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:10:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:10:26.853820 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33793
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:33793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:26.929352 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:26.929580 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:26.929617 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:26.930697 31376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:26.930784 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:26.930809 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:26.930836 31376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:26.930868 31376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:26.932309 31376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:33793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:33793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:26.932580 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:26.932807 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:26.937604 31384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:26.937575 31382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:26.937615 31376 server_base.cc:1034] running on GCE node
W20260927 11:10:26.937584 31381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:26.938033 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:26.938313 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:26.939491 31376 hybrid_clock.cc:648] HybridClock initialized: now 1790507426939486 us; error 50 us; skew 500 ppm
Sep 27 11:10:26 dist-test-slave-6j0v krb5kdc[31363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507426, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:26.947016 31376 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:26.947567 31376 webserver.cc:460] Webserver started at http://127.17.95.254:41957/ using document root <none> and password file <none>
I20260927 11:10:26.947794 31376 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:26.947865 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:26.948017 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:26.949082 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "cfd1baee11a5444ba1cb2a13a662307c"
format_stamp: "Formatted at 2026-09-27 11:10:26 on dist-test-slave-6j0v"
server_key: "1a88e559eab75d3008ced26a8642f9e5"
server_key_iv: "d42792b80f11e237ec35b5d07f3c45c1"
server_key_version: "encryptionkey@0"
I20260927 11:10:26.949522 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "cfd1baee11a5444ba1cb2a13a662307c"
format_stamp: "Formatted at 2026-09-27 11:10:26 on dist-test-slave-6j0v"
server_key: "1a88e559eab75d3008ced26a8642f9e5"
server_key_iv: "d42792b80f11e237ec35b5d07f3c45c1"
server_key_version: "encryptionkey@0"
I20260927 11:10:26.950863 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 11:10:26.951674 31391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:26.951908 31376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:26.952026 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "cfd1baee11a5444ba1cb2a13a662307c"
format_stamp: "Formatted at 2026-09-27 11:10:26 on dist-test-slave-6j0v"
server_key: "1a88e559eab75d3008ced26a8642f9e5"
server_key_iv: "d42792b80f11e237ec35b5d07f3c45c1"
server_key_version: "encryptionkey@0"
I20260927 11:10:26.952112 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:26.965801 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:26.967718 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:26.967895 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:26.971762 31376 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:33793
I20260927 11:10:26.972123 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:26.972127 31443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:33793 every 8 connection(s)
I20260927 11:10:26.972806 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:26.975210 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c: Bootstrap starting.
I20260927 11:10:26.975835 31444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:26.976145 31444 log.cc:824] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:26.976907 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c: No bootstrap required, opened a new log
I20260927 11:10:26.977800 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31376
I20260927 11:10:26.977914 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:10:26.978114 17791 external_mini_cluster.cc:1428] Setting key 30a2cf73c09d771a22e4f840ac68d3cf
I20260927 11:10:26.979408 31444 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1baee11a5444ba1cb2a13a662307c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33793 } }
I20260927 11:10:26.979579 31444 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:26.979640 31444 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd1baee11a5444ba1cb2a13a662307c, State: Initialized, Role: FOLLOWER
I20260927 11:10:26.979750 31444 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1baee11a5444ba1cb2a13a662307c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33793 } }
I20260927 11:10:26.979831 31444 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:26.979861 31444 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:26.979933 31444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:26.980571 31444 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1baee11a5444ba1cb2a13a662307c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33793 } }
I20260927 11:10:26.980691 31444 leader_election.cc:302] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd1baee11a5444ba1cb2a13a662307c; no voters:
I20260927 11:10:26.980863 31444 leader_election.cc:288] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:10:26.980909 31449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:26.981066 31449 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [term 1 LEADER]: Becoming Leader. State: Replica: cfd1baee11a5444ba1cb2a13a662307c, State: Running, Role: LEADER
I20260927 11:10:26.981120 31444 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:26.981221 31449 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1baee11a5444ba1cb2a13a662307c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33793 } }
I20260927 11:10:26.981858 31451 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd1baee11a5444ba1cb2a13a662307c. Latest consensus state: current_term: 1 leader_uuid: "cfd1baee11a5444ba1cb2a13a662307c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1baee11a5444ba1cb2a13a662307c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33793 } } }
I20260927 11:10:26.981933 31451 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:26.982136 31450 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd1baee11a5444ba1cb2a13a662307c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1baee11a5444ba1cb2a13a662307c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 33793 } } }
I20260927 11:10:26.982245 31450 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:26.982388 31458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:26.982913 31458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 11:10:26 dist-test-slave-6j0v krb5kdc[31363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:26.985534 31458 catalog_manager.cc:1347] Generated new cluster ID: 1dad2d834b6e465fbd6528b55c369462
I20260927 11:10:26.985622 31458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:10:26.989786 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:26.978631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.17.95.254:33793)
0927 11:10:26.978845 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:26.978856 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:26.978893 (+ 37us) server_negotiation.cc:407] Connection header received
0927 11:10:26.979375 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:26.979383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:26.979442 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:26.979608 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:26.980416 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:26.981687 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:26.982864 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:26.983290 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:26.983402 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:26.985963 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:26.985967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:26.985968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:26.985979 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:26.988060 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:26.988421 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:26.988424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:26.988425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:26.988537 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:26.988754 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:26.988755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:26.988755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:26.989061 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:26.989210 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:26.989308 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 11:10:26.989440 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260927 11:10:26.990983 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:10:27.001901 31458 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:10:27.002619 31458 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:27.014497 31458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd1baee11a5444ba1cb2a13a662307c: Generated new TSK 0
I20260927 11:10:27.014665 31458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:10:27.044811 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31376
2026-09-27T11:10:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1308 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:10:27 dist-test-slave-6j0v krb5kdc[31471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:10:27 dist-test-slave-6j0v krb5kdc[31471](info): set up 2 sockets
Sep 27 11:10:27 dist-test-slave-6j0v krb5kdc[31471](info): commencing operation
krb5kdc: starting...
W20260927 11:10:28.047216 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:10:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:10:28.158193 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36575
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:36575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:28.233258 31484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:28.233471 31484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:28.233505 31484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:28.234581 31484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:28.234663 31484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:28.234686 31484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:28.234714 31484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:28.234743 31484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:28.236122 31484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:36575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:28.236377 31484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:28.236577 31484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:28.241247 31492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:28.241271 31489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.241432 31484 server_base.cc:1034] running on GCE node
W20260927 11:10:28.241263 31490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.241714 31484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:28.241978 31484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:28.243153 31484 hybrid_clock.cc:648] HybridClock initialized: now 1790507428243130 us; error 32 us; skew 500 ppm
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:28.245358 31484 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:28.245769 31484 webserver.cc:460] Webserver started at http://127.17.95.254:33839/ using document root <none> and password file <none>
I20260927 11:10:28.245944 31484 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:28.245981 31484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:28.246063 31484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:28.246958 31484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "de66082f6b8a466991d05c696dded292"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "067f031f4a93c9895977d2308978d8d4"
server_key_iv: "130996815676b5740d02c4c8229cb911"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.247331 31484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "de66082f6b8a466991d05c696dded292"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "067f031f4a93c9895977d2308978d8d4"
server_key_iv: "130996815676b5740d02c4c8229cb911"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.248564 31484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:28.249198 31499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.249372 31484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:28.249468 31484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "de66082f6b8a466991d05c696dded292"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "067f031f4a93c9895977d2308978d8d4"
server_key_iv: "130996815676b5740d02c4c8229cb911"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.249544 31484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:28.260553 31484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:28.262333 31484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:28.262467 31484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:28.266790 31484 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:36575
I20260927 11:10:28.266815 31551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:36575 every 8 connection(s)
I20260927 11:10:28.267155 31484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:28.268136 31552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:28.270315 31552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap starting.
I20260927 11:10:28.270869 31552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:28.271191 31552 log.cc:824] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:28.271878 31552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: No bootstrap required, opened a new log
I20260927 11:10:28.271906 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31484
I20260927 11:10:28.272003 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:10:28.272217 17791 external_mini_cluster.cc:1428] Setting key 2c55293560b9e3a3735df81aa352f2fe
I20260927 11:10:28.273705 31552 raft_consensus.cc:348] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.273855 31552 raft_consensus.cc:374] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:28.273926 31552 raft_consensus.cc:729] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de66082f6b8a466991d05c696dded292, State: Initialized, Role: FOLLOWER
I20260927 11:10:28.274087 31552 consensus_queue.cc:260] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.274175 31552 raft_consensus.cc:388] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:28.274206 31552 raft_consensus.cc:482] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:28.274255 31552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:28.275295 31552 raft_consensus.cc:504] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.275388 31552 leader_election.cc:302] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de66082f6b8a466991d05c696dded292; no voters:
I20260927 11:10:28.275571 31552 leader_election.cc:288] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:10:28.275624 31557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:28.275759 31557 raft_consensus.cc:686] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 LEADER]: Becoming Leader. State: Replica: de66082f6b8a466991d05c696dded292, State: Running, Role: LEADER
I20260927 11:10:28.275820 31552 sys_catalog.cc:564] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:28.275895 31557 consensus_queue.cc:237] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.277005 31559 sys_catalog.cc:455] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de66082f6b8a466991d05c696dded292. Latest consensus state: current_term: 1 leader_uuid: "de66082f6b8a466991d05c696dded292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } } }
I20260927 11:10:28.277105 31559 sys_catalog.cc:458] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:28.277432 31558 sys_catalog.cc:455] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de66082f6b8a466991d05c696dded292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } } }
I20260927 11:10:28.277678 31558 sys_catalog.cc:458] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: This master's current role is: LEADER
W20260927 11:10:28.277789 31572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:10:28.277837 31572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:10:28.277881 31573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:28.278043 31573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:28.279731 31573 catalog_manager.cc:1347] Generated new cluster ID: fb8a04c6321844e9918344d7f06b3ada
I20260927 11:10:28.279783 31573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:28.284642 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.272673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.17.95.254:36575)
0927 11:10:28.272841 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.272848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.272872 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:10:28.273695 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.273703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.273751 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.273860 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:28.274719 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.275934 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.277265 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.277875 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.277996 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.280773 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:28.280777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:28.280778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:28.280789 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:28.282542 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.282934 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.282937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.282937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.283164 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.283426 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.283427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.283428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.283713 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:28.284266 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.284330 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.284456 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260927 11:10:28.284723 31573 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:10:28.285310 31573 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:28.293239 31573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Generated new TSK 0
I20260927 11:10:28.293393 31573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:10:28.318073 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:36575
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:28.396798 31578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:28.396999 31578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:28.397023 31578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:28.398176 31578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:28.398221 31578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:28.398248 31578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:10:28.399830 31578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:28.400079 31578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:28.400332 31578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:28.405174 31584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:28.405182 31583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:28.405297 31586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.405522 31578 server_base.cc:1034] running on GCE node
I20260927 11:10:28.405718 31578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:28.405912 31578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:28.407107 31578 hybrid_clock.cc:648] HybridClock initialized: now 1790507428407050 us; error 32 us; skew 500 ppm
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:28.409708 31578 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:10:28.410152 31578 webserver.cc:460] Webserver started at http://127.17.95.193:36423/ using document root <none> and password file <none>
I20260927 11:10:28.410378 31578 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:28.410444 31578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:28.410596 31578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:28.411679 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "7468294f1c44411787753b1bdf7702e4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "68c5964b5230d340e24b39577f11c6af"
server_key_iv: "0e77d090091e7777167b791b31faa21e"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.412025 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "7468294f1c44411787753b1bdf7702e4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "68c5964b5230d340e24b39577f11c6af"
server_key_iv: "0e77d090091e7777167b791b31faa21e"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.413300 31578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:10:28.414055 31593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.414263 31578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:28.414359 31578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7468294f1c44411787753b1bdf7702e4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "68c5964b5230d340e24b39577f11c6af"
server_key_iv: "0e77d090091e7777167b791b31faa21e"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.414433 31578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:28.430400 31578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:28.432300 31578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:28.432471 31578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:28.432704 31578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:28.433013 31578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:28.433065 31578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.433099 31578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:28.433125 31578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.438869 31578 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:38549
I20260927 11:10:28.438923 31706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:38549 every 8 connection(s)
I20260927 11:10:28.439273 31578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:10:28.442484 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31578
I20260927 11:10:28.442570 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:10:28.442762 17791 external_mini_cluster.cc:1428] Setting key 42efbc61781af96ac861137d553bec85
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:28.448632 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.440030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45675 (local address 127.17.95.254:36575)
0927 11:10:28.440097 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.440098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.440293 (+ 195us) server_negotiation.cc:407] Connection header received
0927 11:10:28.440776 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.440777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.440791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.440842 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:28.441701 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.442182 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.443376 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.443677 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.443743 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.445751 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:28.445753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:28.445753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:28.445761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:28.447219 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.447652 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.447654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.447654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.447718 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.447974 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.447975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.447975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.448096 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:28.448173 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.448452 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.448541 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:10:28.448829 31709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.440001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:45675)
0927 11:10:28.440169 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.440203 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.440689 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.440896 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.440899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.441423 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:28.441614 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.441625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.442263 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.442264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.443293 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.443296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.443753 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.443754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.443851 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.444308 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:28.444315 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:28.445651 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:28.447320 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:28.447328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:28.447330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:28.447554 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:28.447786 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:28.447787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:28.447788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:28.447905 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:28.448163 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:28.448169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:28.448315 (+ 146us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.448418 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.448572 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260927 11:10:28.449486 31707 heartbeater.cc:347] Connected to a master server at 127.17.95.254:36575
I20260927 11:10:28.449596 31707 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:28.449769 31707 heartbeater.cc:511] Master 127.17.95.254:36575 requested a full tablet report, sending...
I20260927 11:10:28.450320 31516 ts_manager.cc:194] Registered new tserver with Master: 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549)
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
I20260927 11:10:28.451299 31516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:45675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:10:28.467939 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:36575
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:28.548485 31712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:28.548683 31712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:28.548718 31712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:28.549829 31712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:28.549867 31712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:28.549892 31712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:10:28.551280 31712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:28.551548 31712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:28.551800 31712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:28.556603 31717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.556766 31712 server_base.cc:1034] running on GCE node
W20260927 11:10:28.556761 31718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:28.556660 31720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.557119 31712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:28.557391 31712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:28.558547 31712 hybrid_clock.cc:648] HybridClock initialized: now 1790507428558545 us; error 40 us; skew 500 ppm
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:28.561079 31712 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:10:28.561553 31712 webserver.cc:460] Webserver started at http://127.17.95.194:38247/ using document root <none> and password file <none>
I20260927 11:10:28.561790 31712 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:28.561853 31712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:28.562014 31712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:28.562999 31712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "7bc8c97ca05347d49a9936b39ff62403"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "2a5bc9d52a8cbd8315fc8873949b178a"
server_key_iv: "c85e693f81b6af505d374f5f777ce3ff"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.563416 31712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "7bc8c97ca05347d49a9936b39ff62403"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "2a5bc9d52a8cbd8315fc8873949b178a"
server_key_iv: "c85e693f81b6af505d374f5f777ce3ff"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.564677 31712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:28.565371 31727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.565560 31712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:28.565650 31712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "7bc8c97ca05347d49a9936b39ff62403"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "2a5bc9d52a8cbd8315fc8873949b178a"
server_key_iv: "c85e693f81b6af505d374f5f777ce3ff"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.565742 31712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:28.574028 31712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:28.575989 31712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:28.576143 31712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:28.576373 31712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:28.576689 31712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:28.576742 31712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.576782 31712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:28.576807 31712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.582484 31712 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:39257
I20260927 11:10:28.582521 31840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:39257 every 8 connection(s)
I20260927 11:10:28.582868 31712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:28.591753 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.583603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:58763 (local address 127.17.95.254:36575)
0927 11:10:28.583687 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.583689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.583836 (+ 147us) server_negotiation.cc:407] Connection header received
0927 11:10:28.584327 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.584328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.584340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.584396 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:28.585278 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.585780 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.586767 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.586973 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.587054 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.589239 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:28.589240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:28.589241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:28.589247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:28.590527 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.590906 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.590907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.590907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.590958 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.591235 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.591235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.591236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.591335 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:28.591387 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.591631 (+ 244us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.591691 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:10:28.591902 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.583545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:58763)
0927 11:10:28.583717 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.583754 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.584206 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.584458 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.584461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.584997 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:28.585172 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.585176 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.585867 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.585869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.586664 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.586666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.587061 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.587062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.587214 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.587744 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:28.587753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:28.589148 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:28.590604 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:28.590610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:28.590611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:28.590837 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:28.591017 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:28.591018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:28.591018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:28.591179 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:28.591392 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:28.591396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:28.591503 (+ 107us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.591604 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.591714 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260927 11:10:28.592478 31841 heartbeater.cc:347] Connected to a master server at 127.17.95.254:36575
I20260927 11:10:28.592532 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31712
I20260927 11:10:28.592612 31841 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:28.592631 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:10:28.592805 31841 heartbeater.cc:511] Master 127.17.95.254:36575 requested a full tablet report, sending...
I20260927 11:10:28.592844 17791 external_mini_cluster.cc:1428] Setting key 0071e3ff00a697a93fd6a259beb13da0
I20260927 11:10:28.593166 31516 ts_manager.cc:194] Registered new tserver with Master: 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
I20260927 11:10:28.593693 31516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:58763
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:10:28.615829 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:36575
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:28.693885 31846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:28.694073 31846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:28.694094 31846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:28.695204 31846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:28.695242 31846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:28.695291 31846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:10:28.696662 31846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:28.696897 31846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:28.697108 31846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:28.701930 31854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.701937 31846 server_base.cc:1034] running on GCE node
W20260927 11:10:28.701905 31851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:28.701910 31852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.702430 31846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:28.702636 31846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:28.703794 31846 hybrid_clock.cc:648] HybridClock initialized: now 1790507428703770 us; error 37 us; skew 500 ppm
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:28.706138 31846 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:10:28.706637 31846 webserver.cc:460] Webserver started at http://127.17.95.195:39487/ using document root <none> and password file <none>
I20260927 11:10:28.706851 31846 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:28.706915 31846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:28.707058 31846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:28.708073 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "a812cfd376594cbabadf775b852684c4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "1add9e9892b96c945050b65b09c013f5"
server_key_iv: "a8748b036a7d66140399b061f648769e"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.708360 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "a812cfd376594cbabadf775b852684c4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "1add9e9892b96c945050b65b09c013f5"
server_key_iv: "a8748b036a7d66140399b061f648769e"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.709625 31846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:10:28.710287 31861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.710438 31846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:10:28.710505 31846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "a812cfd376594cbabadf775b852684c4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "1add9e9892b96c945050b65b09c013f5"
server_key_iv: "a8748b036a7d66140399b061f648769e"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.710568 31846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:28.721227 31846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:28.723008 31846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:28.723181 31846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:28.723354 31846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:28.723594 31846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:28.723624 31846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.723645 31846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:28.723658 31846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.729485 31846 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35243
I20260927 11:10:28.729532 31974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35243 every 8 connection(s)
I20260927 11:10:28.729827 31846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:10:28.730163 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 31846
I20260927 11:10:28.730283 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:10:28.730474 17791 external_mini_cluster.cc:1428] Setting key 30f7b4b2b89346be7a7a9c7123ea39df
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:28.738994 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.730581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:59751 (local address 127.17.95.254:36575)
0927 11:10:28.730629 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.730631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.730922 (+ 291us) server_negotiation.cc:407] Connection header received
0927 11:10:28.731461 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.731463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.731474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.731562 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:28.732482 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.732985 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.733953 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.734146 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.734226 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.736390 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:28.736392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:28.736392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:28.736400 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:28.737663 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.738060 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.738061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.738062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.738122 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.738361 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.738362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.738363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.738477 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:28.738561 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.738828 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.738906 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:10:28.739243 31977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.730526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:59751)
0927 11:10:28.730757 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.730796 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.731357 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.731643 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.731647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.732196 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:28.732373 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.732378 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.733078 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.733079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.733850 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.733852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.734237 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.734238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.734360 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.734804 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:28.734812 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:28.736291 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:28.737746 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:28.737752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:28.737754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:28.737970 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:28.738190 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:28.738191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:28.738191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:28.738294 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:28.738544 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:28.738551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:28.738686 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.738806 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.738955 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260927 11:10:28.739703 31975 heartbeater.cc:347] Connected to a master server at 127.17.95.254:36575
I20260927 11:10:28.739818 31975 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:28.739948 31975 heartbeater.cc:511] Master 127.17.95.254:36575 requested a full tablet report, sending...
I20260927 11:10:28.740283 31516 ts_manager.cc:194] Registered new tserver with Master: a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243)
I20260927 11:10:28.740614 31516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:59751
I20260927 11:10:28.742126 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:10:28.748695 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.743403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.17.95.254:36575)
0927 11:10:28.743459 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.743461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.743567 (+ 106us) server_negotiation.cc:407] Connection header received
0927 11:10:28.743607 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.743608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.743615 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.743664 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:28.744140 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.744569 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.745237 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.745400 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.745449 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.746457 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:28.746458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:28.746458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:28.746464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:28.747821 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.748105 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.748105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.748106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.748170 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:28.748318 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:28.748318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:28.748319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:28.748431 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:28.748493 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.748591 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.748642 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 11:10:28.750046 31516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:10:28.751626 31516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:10:28.757766 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.753762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35243 (local address 127.0.0.1:48202)
0927 11:10:28.754007 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.754016 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.754056 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.754347 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.754350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.754352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:28.754431 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.754432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.755754 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.755755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.756786 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.756788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.757414 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.757414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.757498 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.757538 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.757610 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.757679 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":66,"threads_started":1}
I20260927 11:10:28.757771 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.753934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:39257 (local address 127.0.0.1:43864)
0927 11:10:28.754046 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.754057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.754107 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.754497 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.754497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.754500 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:28.754553 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.754556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.755515 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.755516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.756700 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.756703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.757554 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.757555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.757616 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.757619 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.757676 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.757712 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260927 11:10:28.757894 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.753717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:38549 (local address 127.0.0.1:40936)
0927 11:10:28.753893 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.753935 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.753998 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.754428 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.754428 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.754430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:28.754471 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.754472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.755781 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.755782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.756640 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.756643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.757643 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.757644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.757721 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.757725 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.757774 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.757810 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260927 11:10:28.758107 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.753844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.17.95.195:35243)
0927 11:10:28.754175 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.754180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.754188 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:10:28.754222 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.754228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.754241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.754329 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:28.754515 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.755655 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.756851 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.757318 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.757414 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.757467 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.757977 (+ 510us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.758030 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":86,"threads_started":1}
I20260927 11:10:28.758131 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.753841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.17.95.193:38549)
0927 11:10:28.754203 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.754208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.754219 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:10:28.754259 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.754261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.754276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.754524 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:28.754609 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.755685 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.756738 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.757449 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.757548 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.757608 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.758015 (+ 407us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.758057 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":86,"threads_started":1}
I20260927 11:10:28.758432 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.754147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.17.95.194:39257)
0927 11:10:28.754335 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.754339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.754349 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:10:28.754383 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.754385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.754398 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.754640 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:28.754714 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.755421 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.756789 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.757465 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.757552 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.757604 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.757690 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.757738 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260927 11:10:28.758765 31641 tablet_service.cc:1467] Processing CreateTablet for tablet 5393296f7ab84ebab1620ec860602b4b (DEFAULT_TABLE table=test-table [id=059d97eb23e34dba840b4d4d60a2ddce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:28.758762 31909 tablet_service.cc:1467] Processing CreateTablet for tablet 5393296f7ab84ebab1620ec860602b4b (DEFAULT_TABLE table=test-table [id=059d97eb23e34dba840b4d4d60a2ddce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:28.759105 31909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5393296f7ab84ebab1620ec860602b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:28.759156 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5393296f7ab84ebab1620ec860602b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:28.759032 31775 tablet_service.cc:1467] Processing CreateTablet for tablet 5393296f7ab84ebab1620ec860602b4b (DEFAULT_TABLE table=test-table [id=059d97eb23e34dba840b4d4d60a2ddce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:28.759351 31775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5393296f7ab84ebab1620ec860602b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:28.761816 31989 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Bootstrap starting.
I20260927 11:10:28.762244 31990 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Bootstrap starting.
I20260927 11:10:28.762488 31991 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Bootstrap starting.
I20260927 11:10:28.762663 31989 tablet_bootstrap.cc:654] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:28.762892 31990 tablet_bootstrap.cc:654] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:28.763180 31991 tablet_bootstrap.cc:654] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:28.763286 31989 log.cc:824] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:28.763491 31990 log.cc:824] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:28.763780 31991 log.cc:824] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:28.764134 31989 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: No bootstrap required, opened a new log
I20260927 11:10:28.764143 31990 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: No bootstrap required, opened a new log
I20260927 11:10:28.764206 31989 ts_tablet_manager.cc:1397] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:10:28.764206 31990 ts_tablet_manager.cc:1397] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:10:28.764499 31991 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: No bootstrap required, opened a new log
I20260927 11:10:28.764576 31991 ts_tablet_manager.cc:1397] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:10:28.766058 31991 raft_consensus.cc:348] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.766216 31991 raft_consensus.cc:374] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:28.766258 31991 raft_consensus.cc:729] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7468294f1c44411787753b1bdf7702e4, State: Initialized, Role: FOLLOWER
I20260927 11:10:28.766177 31990 raft_consensus.cc:348] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.766177 31989 raft_consensus.cc:348] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.766328 31989 raft_consensus.cc:374] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:28.766328 31990 raft_consensus.cc:374] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:28.766369 31990 raft_consensus.cc:729] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a812cfd376594cbabadf775b852684c4, State: Initialized, Role: FOLLOWER
I20260927 11:10:28.766369 31989 raft_consensus.cc:729] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc8c97ca05347d49a9936b39ff62403, State: Initialized, Role: FOLLOWER
I20260927 11:10:28.766395 31991 consensus_queue.cc:260] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.766484 31989 consensus_queue.cc:260] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.766497 31990 consensus_queue.cc:260] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.766665 31991 ts_tablet_manager.cc:1428] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:10:28.766754 31990 ts_tablet_manager.cc:1428] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:10:28.766754 31989 ts_tablet_manager.cc:1428] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:10:28.766780 31707 heartbeater.cc:503] Master 127.17.95.254:36575 was elected leader, sending a full tablet report...
I20260927 11:10:28.766788 31975 heartbeater.cc:503] Master 127.17.95.254:36575 was elected leader, sending a full tablet report...
I20260927 11:10:28.766865 31841 heartbeater.cc:503] Master 127.17.95.254:36575 was elected leader, sending a full tablet report...
I20260927 11:10:28.786763 31996 raft_consensus.cc:482] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:28.786871 31996 raft_consensus.cc:504] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.787349 31996 leader_election.cc:288] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
I20260927 11:10:28.790293 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.787660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:48929 (local address 127.17.95.194:39257)
0927 11:10:28.787705 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.787707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.787819 (+ 112us) server_negotiation.cc:407] Connection header received
0927 11:10:28.787867 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.787869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.787880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.787938 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:28.788083 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.788670 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.789599 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.789948 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.790050 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.790058 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.790188 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.790226 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:10:28.790298 31977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.787488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:38549 (local address 127.17.95.195:39313)
0927 11:10:28.787543 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.787574 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.787623 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.787768 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.787769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.787770 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:28.787865 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.787867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.788712 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.788713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.789467 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.789469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.790042 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.790043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.790123 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.790126 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.790195 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.790229 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":22}
I20260927 11:10:28.790299 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.787582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:39313 (local address 127.17.95.193:38549)
0927 11:10:28.787636 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.787638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.787650 (+ 12us) server_negotiation.cc:407] Connection header received
0927 11:10:28.787680 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.787680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.787691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.787757 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:28.787952 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.788571 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.789556 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.789940 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.790005 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.790031 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.790207 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.790245 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 11:10:28.790313 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.787494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:39257 (local address 127.17.95.195:48929)
0927 11:10:28.787746 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.787756 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:28.787812 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:28.787953 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:28.787953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:28.787955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:28.788016 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.788018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.788761 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.788761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.789485 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:28.789487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.790069 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.790069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:28.790121 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.790124 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:28.790163 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 11:10:28.790219 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":12,"thread_start_us":118,"threads_started":1}
I20260927 11:10:28.790598 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5393296f7ab84ebab1620ec860602b4b" candidate_uuid: "a812cfd376594cbabadf775b852684c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7468294f1c44411787753b1bdf7702e4" is_pre_election: true
I20260927 11:10:28.790758 31661 raft_consensus.cc:2454] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a812cfd376594cbabadf775b852684c4 in term 0.
I20260927 11:10:28.790693 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5393296f7ab84ebab1620ec860602b4b" candidate_uuid: "a812cfd376594cbabadf775b852684c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc8c97ca05347d49a9936b39ff62403" is_pre_election: true
I20260927 11:10:28.790870 31795 raft_consensus.cc:2454] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a812cfd376594cbabadf775b852684c4 in term 0.
I20260927 11:10:28.791019 31864 leader_election.cc:302] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4, a812cfd376594cbabadf775b852684c4; no voters:
I20260927 11:10:28.791163 31996 raft_consensus.cc:2781] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:10:28.791234 31996 raft_consensus.cc:482] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:10:28.791253 31996 raft_consensus.cc:3037] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:28.791975 31996 raft_consensus.cc:504] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.792135 31996 leader_election.cc:288] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
I20260927 11:10:28.792289 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5393296f7ab84ebab1620ec860602b4b" candidate_uuid: "a812cfd376594cbabadf775b852684c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7468294f1c44411787753b1bdf7702e4"
I20260927 11:10:28.792382 31661 raft_consensus.cc:3037] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:28.793109 31661 raft_consensus.cc:2454] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a812cfd376594cbabadf775b852684c4 in term 1.
I20260927 11:10:28.793256 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5393296f7ab84ebab1620ec860602b4b" candidate_uuid: "a812cfd376594cbabadf775b852684c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc8c97ca05347d49a9936b39ff62403"
I20260927 11:10:28.793360 31864 leader_election.cc:302] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4, a812cfd376594cbabadf775b852684c4; no voters:
I20260927 11:10:28.793371 31795 raft_consensus.cc:3037] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:28.793464 31996 raft_consensus.cc:2781] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:28.793642 31996 raft_consensus.cc:686] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 1 LEADER]: Becoming Leader. State: Replica: a812cfd376594cbabadf775b852684c4, State: Running, Role: LEADER
I20260927 11:10:28.793771 31996 consensus_queue.cc:237] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:28.794271 31795 raft_consensus.cc:2454] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a812cfd376594cbabadf775b852684c4 in term 1.
I20260927 11:10:28.794598 31514 catalog_manager.cc:5495] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a812cfd376594cbabadf775b852684c4 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "a812cfd376594cbabadf775b852684c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } health_report { overall_health: UNKNOWN } } }
I20260927 11:10:28.802276 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.800072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.17.95.195:35243)
0927 11:10:28.800112 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:28.800114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:28.800123 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:10:28.800160 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:28.800161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:28.800175 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:28.800239 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:10:28.800339 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.800808 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.801508 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:28.801698 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:28.801752 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:28.801963 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:10:28.802055 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:10:28.802119 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:28.802162 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 11:10:28.802179 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:10:28.804291 31795 raft_consensus.cc:1261] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Refusing update from remote peer a812cfd376594cbabadf775b852684c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:28.804388 31661 raft_consensus.cc:1261] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Refusing update from remote peer a812cfd376594cbabadf775b852684c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:28.804685 31999 consensus_queue.cc:1035] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:28.804849 31996 consensus_queue.cc:1035] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:28.807017 32002 mvcc.cc:205] Tried to move back new op lower bound from 7333918428379959296 to 7333918428339892224. Current Snapshot: MvccSnapshot[applied={T|T < 7333918428379959296}]
I20260927 11:10:28.807370 32004 mvcc.cc:205] Tried to move back new op lower bound from 7333918428379959296 to 7333918428339892224. Current Snapshot: MvccSnapshot[applied={T|T < 7333918428379959296}]
I20260927 11:10:28.807969 32003 mvcc.cc:205] Tried to move back new op lower bound from 7333918428379959296 to 7333918428339892224. Current Snapshot: MvccSnapshot[applied={T|T < 7333918428379959296 or (T in {7333918428379959296})}]
I20260927 11:10:28.810256 31514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:28.810362 31514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:28.811115 31514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Sending DeleteTablet for 3 replicas of tablet 5393296f7ab84ebab1620ec860602b4b
I20260927 11:10:28.811442 31641 tablet_service.cc:1514] Processing DeleteTablet for tablet 5393296f7ab84ebab1620ec860602b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:28 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:40936
I20260927 11:10:28.811451 31775 tablet_service.cc:1514] Processing DeleteTablet for tablet 5393296f7ab84ebab1620ec860602b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:28 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43864
I20260927 11:10:28.811530 31909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5393296f7ab84ebab1620ec860602b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:28 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:48202
I20260927 11:10:28.811739 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31578
I20260927 11:10:28.811756 32013 tablet_replica.cc:330] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: stopping tablet replica
I20260927 11:10:28.812371 32012 tablet_replica.cc:330] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: stopping tablet replica
I20260927 11:10:28.811764 32014 tablet_replica.cc:330] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: stopping tablet replica
I20260927 11:10:28.812556 32012 raft_consensus.cc:2229] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:28.812668 32012 raft_consensus.cc:2258] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:28.812659 32014 raft_consensus.cc:2229] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:10:28.812832 32014 raft_consensus.cc:2258] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:28.813211 32012 ts_tablet_manager.cc:1905] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:28.813503 32014 ts_tablet_manager.cc:1905] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:28.814975 32012 ts_tablet_manager.cc:1918] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:28.815068 32012 log.cc:1196] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/5393296f7ab84ebab1620ec860602b4b
I20260927 11:10:28.815412 32012 ts_tablet_manager.cc:1939] T 5393296f7ab84ebab1620ec860602b4b P 7bc8c97ca05347d49a9936b39ff62403: Deleting consensus metadata
I20260927 11:10:28.815714 32014 ts_tablet_manager.cc:1918] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:28.815836 32014 log.cc:1196] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/5393296f7ab84ebab1620ec860602b4b
I20260927 11:10:28.815842 31501 catalog_manager.cc:4841] TS 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): tablet 5393296f7ab84ebab1620ec860602b4b (table test-table [id=059d97eb23e34dba840b4d4d60a2ddce]) successfully deleted
I20260927 11:10:28.816195 32014 ts_tablet_manager.cc:1939] T 5393296f7ab84ebab1620ec860602b4b P a812cfd376594cbabadf775b852684c4: Deleting consensus metadata
I20260927 11:10:28.816612 31500 catalog_manager.cc:4841] TS a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): tablet 5393296f7ab84ebab1620ec860602b4b (table test-table [id=059d97eb23e34dba840b4d4d60a2ddce]) successfully deleted
W20260927 11:10:28.818599 31502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:38549 (error 108)
I20260927 11:10:28.818760 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31712
I20260927 11:10:28.819603 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:28.819329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:38549 (local address 127.0.0.1:40938)
0927 11:10:28.819541 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:28.819578 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
W20260927 11:10:28.819726 31502 catalog_manager.cc:4584] TS 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5393296f7ab84ebab1620ec860602b4b: Network error: Client connection negotiation failed: client connection to 127.17.95.193:38549: connect: Connection refused (error 111)
I20260927 11:10:28.824661 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31846
I20260927 11:10:28.830129 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 31484
I20260927 11:10:28.835696 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36575
--webserver_interface=127.17.95.254
--webserver_port=33839
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:36575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:28.914074 32016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:28.914278 32016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:28.914318 32016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:28.915395 32016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:28.915459 32016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:28.915482 32016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:28.915504 32016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:28.915531 32016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:28.916927 32016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:36575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=33839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:28.917189 32016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:28.917402 32016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:28.925701 32025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:28.925740 32023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.925876 32016 server_base.cc:1034] running on GCE node
W20260927 11:10:28.925938 32022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:28.926227 32016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:28.926535 32016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:28.927723 32016 hybrid_clock.cc:648] HybridClock initialized: now 1790507428927709 us; error 39 us; skew 500 ppm
Sep 27 11:10:28 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:28.930088 32016 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:28.930509 32016 webserver.cc:460] Webserver started at http://127.17.95.254:33839/ using document root <none> and password file <none>
I20260927 11:10:28.930706 32016 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:28.930753 32016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:28.932068 32016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:28.932709 32032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:28.932883 32016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:28.932955 32016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "de66082f6b8a466991d05c696dded292"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "067f031f4a93c9895977d2308978d8d4"
server_key_iv: "130996815676b5740d02c4c8229cb911"
server_key_version: "encryptionkey@0"
I20260927 11:10:28.933314 32016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:28.943660 32016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:28.945459 32016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:28.945587 32016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:28.949360 32016 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:36575
I20260927 11:10:28.949429 32084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:36575 every 8 connection(s)
I20260927 11:10:28.949829 32016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:28.951442 32085 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:10:28.952072 32085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap starting.
I20260927 11:10:28.959997 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 32016
I20260927 11:10:28.960140 32085 log.cc:824] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:28.960316 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:38549
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:36575
--webserver_port=36423
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:10:28.962037 32085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:10:28.962343 32085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap complete.
I20260927 11:10:28.964586 32085 raft_consensus.cc:348] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.964808 32085 raft_consensus.cc:729] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de66082f6b8a466991d05c696dded292, State: Initialized, Role: FOLLOWER
I20260927 11:10:28.964972 32085 consensus_queue.cc:260] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.965044 32085 raft_consensus.cc:388] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:28.965098 32085 raft_consensus.cc:482] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:28.965143 32085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 1 FOLLOWER]: Advancing to term 2
I20260927 11:10:28.966074 32085 raft_consensus.cc:504] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.966184 32085 leader_election.cc:302] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de66082f6b8a466991d05c696dded292; no voters:
I20260927 11:10:28.966372 32085 leader_election.cc:288] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 11:10:28.966405 32089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 11:10:28.966506 32089 raft_consensus.cc:686] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 LEADER]: Becoming Leader. State: Replica: de66082f6b8a466991d05c696dded292, State: Running, Role: LEADER
I20260927 11:10:28.966590 32089 consensus_queue.cc:237] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:28.966614 32085 sys_catalog.cc:564] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:28.966826 32090 sys_catalog.cc:455] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "de66082f6b8a466991d05c696dded292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } } }
I20260927 11:10:28.966919 32090 sys_catalog.cc:458] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:28.967139 32091 sys_catalog.cc:455] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de66082f6b8a466991d05c696dded292. Latest consensus state: current_term: 2 leader_uuid: "de66082f6b8a466991d05c696dded292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } } }
I20260927 11:10:28.967224 32091 sys_catalog.cc:458] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:28.967136 32093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:28.968109 32093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:28.968222 32093 catalog_manager.cc:1259] Loaded cluster ID: fb8a04c6321844e9918344d7f06b3ada
I20260927 11:10:28.968488 32093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:10:28.968713 32093 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 11:10:28.971449 32093 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 11:10:28.973101 32093 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:28.973294 32093 catalog_manager.cc:5879] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Loaded TSK: 0
I20260927 11:10:28.973507 32093 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 11:10:29.055001 32088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:29.055272 32088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:29.055295 32088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:29.056447 32088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:29.056488 32088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:29.056514 32088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:10:29.057930 32088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:38549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=36423
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:29.058151 32088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:29.058391 32088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:29.064132 32112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.064182 32113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.064247 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:29.064386 32088 server_base.cc:1034] running on GCE node
I20260927 11:10:29.064635 32088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:29.064918 32088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:29.066072 32088 hybrid_clock.cc:648] HybridClock initialized: now 1790507429066059 us; error 29 us; skew 500 ppm
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:29.068885 32088 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:10:29.069429 32088 webserver.cc:460] Webserver started at http://127.17.95.193:36423/ using document root <none> and password file <none>
I20260927 11:10:29.069684 32088 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:29.069773 32088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:29.071336 32088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:10:29.072113 32122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.072338 32088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:29.072454 32088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7468294f1c44411787753b1bdf7702e4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "68c5964b5230d340e24b39577f11c6af"
server_key_iv: "0e77d090091e7777167b791b31faa21e"
server_key_version: "encryptionkey@0"
I20260927 11:10:29.072844 32088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:29.092355 32088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:29.094548 32088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:29.094738 32088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:29.094992 32088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:29.095611 32129 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 11:10:29.097400 32088 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 11:10:29.097478 32088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 11:10:29.097525 32088 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 11:10:29.098169 32088 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 11:10:29.098222 32088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:29.098281 32129 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Bootstrap starting.
I20260927 11:10:29.105324 32088 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:38549
I20260927 11:10:29.105396 32236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:38549 every 8 connection(s)
I20260927 11:10:29.105729 32088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:10:29.109073 32129 log.cc:824] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:29.111507 32129 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:10:29.111828 32129 tablet_bootstrap.cc:492] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Bootstrap complete.
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:29.113013 32129 ts_tablet_manager.cc:1397] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260927 11:10:29.114049 32129 raft_consensus.cc:348] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:29.114315 32129 raft_consensus.cc:729] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7468294f1c44411787753b1bdf7702e4, State: Initialized, Role: FOLLOWER
I20260927 11:10:29.114437 32129 consensus_queue.cc:260] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } } peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } }
I20260927 11:10:29.114642 32129 ts_tablet_manager.cc:1428] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:10:29.115710 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 32088
I20260927 11:10:29.115962 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:39257
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:36575
--webserver_port=38247
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:10:29.116425 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.106420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:37129 (local address 127.17.95.254:36575)
0927 11:10:29.107327 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.107334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.107357 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:10:29.107457 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.107462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.107499 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.107639 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:29.108243 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.109080 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.110267 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.110640 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.110762 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.112997 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:29.113000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:29.113001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:29.113011 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:29.114791 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.115213 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.115215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.115216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.115310 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.115553 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.115560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.115560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.115797 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:29.115864 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.116182 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.116282 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":70,"threads_started":1}
I20260927 11:10:29.116796 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.106368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:37129)
0927 11:10:29.106635 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.106669 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.107122 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.107665 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.107668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.107952 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:29.108129 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.108133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.109218 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.109219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.110166 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.110169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.110755 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.110756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.110899 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.111423 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:29.111430 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:29.112889 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:29.114871 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.114877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.114879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.115122 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.115373 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.115374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.115374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.115476 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.115857 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:29.115862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:29.115982 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.116530 (+ 548us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.116683 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":81,"threads_started":1}
I20260927 11:10:29.117532 32237 heartbeater.cc:347] Connected to a master server at 127.17.95.254:36575
I20260927 11:10:29.117641 32237 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:29.117844 32237 heartbeater.cc:511] Master 127.17.95.254:36575 requested a full tablet report, sending...
I20260927 11:10:29.118525 32049 ts_manager.cc:194] Registered new tserver with Master: 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549)
I20260927 11:10:29.119511 32049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:37129
I20260927 11:10:29.120579 32237 heartbeater.cc:503] Master 127.17.95.254:36575 was elected leader, sending a full tablet report...
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507428, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.193@KRBTEST.COM
I20260927 11:10:29.127336 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.119784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.17.95.193:38549)
0927 11:10:29.119918 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.119922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.119931 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:10:29.119960 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.119962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.119977 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.120054 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:29.120417 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.121246 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.121934 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.122238 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.122314 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.124533 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:29.124539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:29.124539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:29.124544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:29.125983 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.126469 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.126471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.126471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.126518 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.126711 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.126712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.126712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.126820 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:29.126894 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.127199 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.127268 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260927 11:10:29.127382 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.119099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:38549 (local address 127.0.0.1:40950)
0927 11:10:29.119253 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.119290 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.119351 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.120057 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.120062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.120299 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:29.120337 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.120339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.121316 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.121317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.121841 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.121843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.122313 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.122314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.122391 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.122835 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:29.122839 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:29.124453 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:29.126074 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.126081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.126082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.126375 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.126579 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.126580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.126580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.126649 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.126908 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:29.126915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:29.126971 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.127161 (+ 190us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.127272 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260927 11:10:29.127758 32171 tablet_service.cc:1514] Processing DeleteTablet for tablet 5393296f7ab84ebab1620ec860602b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:28 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:40950
I20260927 11:10:29.128180 32246 tablet_replica.cc:330] stopping tablet replica
I20260927 11:10:29.128278 32246 raft_consensus.cc:2229] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:29.128342 32246 raft_consensus.cc:2258] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:29.128597 32246 ts_tablet_manager.cc:1905] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:29.130406 32246 ts_tablet_manager.cc:1918] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:29.130466 32246 log.cc:1196] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/5393296f7ab84ebab1620ec860602b4b
I20260927 11:10:29.130692 32246 ts_tablet_manager.cc:1939] T 5393296f7ab84ebab1620ec860602b4b P 7468294f1c44411787753b1bdf7702e4: Deleting consensus metadata
I20260927 11:10:29.131170 32035 catalog_manager.cc:4841] TS 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549): tablet 5393296f7ab84ebab1620ec860602b4b (table test-table [id=059d97eb23e34dba840b4d4d60a2ddce]) successfully deleted
W20260927 11:10:29.198678 32243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:29.198922 32243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:29.198956 32243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:29.200057 32243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:29.200119 32243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:29.200165 32243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:10:29.201592 32243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:39257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=38247
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:29.201822 32243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:29.202023 32243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:29.206727 32254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.206712 32252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.206704 32251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:29.206995 32243 server_base.cc:1034] running on GCE node
I20260927 11:10:29.207224 32243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:29.207464 32243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:29.208598 32243 hybrid_clock.cc:648] HybridClock initialized: now 1790507429208598 us; error 38 us; skew 500 ppm
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:29.210958 32243 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:10:29.211378 32243 webserver.cc:460] Webserver started at http://127.17.95.194:38247/ using document root <none> and password file <none>
I20260927 11:10:29.211603 32243 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:29.211683 32243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:29.212898 32243 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:29.213508 32261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.213693 32243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:29.213775 32243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "7bc8c97ca05347d49a9936b39ff62403"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "2a5bc9d52a8cbd8315fc8873949b178a"
server_key_iv: "c85e693f81b6af505d374f5f777ce3ff"
server_key_version: "encryptionkey@0"
I20260927 11:10:29.214076 32243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:29.227149 32243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:29.229305 32243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:29.229449 32243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:29.229655 32243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:29.229948 32243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:29.230000 32243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.230038 32243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:29.230093 32243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.235627 32243 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:39257
I20260927 11:10:29.235674 32374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:39257 every 8 connection(s)
I20260927 11:10:29.235980 32243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:10:29.241276 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 32243
I20260927 11:10:29.241613 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:35243
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:36575
--webserver_port=39487
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:29.245800 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.236765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:59967 (local address 127.17.95.254:36575)
0927 11:10:29.236842 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.236844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.237007 (+ 163us) server_negotiation.cc:407] Connection header received
0927 11:10:29.237504 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.237505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.237517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.237619 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:29.238460 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.238963 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.239949 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.240165 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.240230 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.242655 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:29.242656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:29.242657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:29.242664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:29.244234 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.244758 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.244759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.244759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.244815 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.245116 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.245116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.245117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.245218 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:29.245271 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.245646 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.245732 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:10:29.246068 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.236752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:59967)
0927 11:10:29.236888 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.236925 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.237403 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.237671 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.237674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.238200 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:29.238371 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.238376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.239060 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.239061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.239859 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.239861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.240244 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.240245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.240341 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.240823 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:29.240831 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:29.242530 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:29.244327 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.244333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.244336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.244663 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.244891 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.244891 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.244892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.245046 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.245289 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:29.245295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:29.245457 (+ 162us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.245581 (+ 124us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.245762 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260927 11:10:29.246642 32375 heartbeater.cc:347] Connected to a master server at 127.17.95.254:36575
I20260927 11:10:29.246752 32375 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:29.246956 32375 heartbeater.cc:511] Master 127.17.95.254:36575 requested a full tablet report, sending...
I20260927 11:10:29.247464 32049 ts_manager.cc:194] Registered new tserver with Master: 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
I20260927 11:10:29.247848 32049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:59967
W20260927 11:10:29.322628 32378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:29.322818 32378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:29.322858 32378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:29.323999 32378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:29.324040 32378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:29.324095 32378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:10:29.325471 32378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:35243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=39487
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:29.325698 32378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:29.325904 32378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:29.330574 32383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.330601 32384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.330614 32386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:29.330783 32378 server_base.cc:1034] running on GCE node
I20260927 11:10:29.330972 32378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:29.331218 32378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:29.332418 32378 hybrid_clock.cc:648] HybridClock initialized: now 1790507429332405 us; error 26 us; skew 500 ppm
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:29.334794 32378 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:10:29.335282 32378 webserver.cc:460] Webserver started at http://127.17.95.195:39487/ using document root <none> and password file <none>
I20260927 11:10:29.335492 32378 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:29.335567 32378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:29.336910 32378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:29.337563 32393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.337885 32378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:29.337992 32378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "a812cfd376594cbabadf775b852684c4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "1add9e9892b96c945050b65b09c013f5"
server_key_iv: "a8748b036a7d66140399b061f648769e"
server_key_version: "encryptionkey@0"
I20260927 11:10:29.338356 32378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:29.345256 32378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:29.347555 32378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:29.347735 32378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:29.347951 32378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:29.348244 32378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:29.348300 32378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.348343 32378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:29.348387 32378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.354635 32378 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35243
I20260927 11:10:29.354666 32506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35243 every 8 connection(s)
I20260927 11:10:29.354995 32378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:10:29.357052 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 32378
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:29.365087 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.355762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:39331 (local address 127.17.95.254:36575)
0927 11:10:29.355840 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.355843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.356012 (+ 169us) server_negotiation.cc:407] Connection header received
0927 11:10:29.356446 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.356448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.356458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.356516 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:29.357449 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.358044 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.359381 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.359678 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.359756 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.362020 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:29.362022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:29.362022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:29.362048 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:29.363829 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.364204 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.364205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.364206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.364260 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.364505 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.364505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.364506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.364612 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:29.364675 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.364924 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.364996 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 11:10:29.365244 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.355734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:39331)
0927 11:10:29.355899 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.355930 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.356355 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.356563 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.356566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.357127 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:29.357346 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.357352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.358125 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.358126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.359310 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.359313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.359779 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.359780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.359945 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.360389 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:29.360398 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:29.361929 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:29.363911 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.363917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.363919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.364130 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.364343 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:29.364344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:29.364344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:29.364437 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:29.364677 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:29.364684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:29.364802 (+ 118us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.364893 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.365025 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260927 11:10:29.365331 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.358145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.17.95.254:36575)
0927 11:10:29.358261 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.358263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.358273 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:10:29.358297 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.358297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.358307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.358363 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:29.359126 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.359758 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.360786 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.360976 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.361092 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.362438 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:29.362439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:29.362440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:29.362446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:29.363973 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.364389 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.364391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.364391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.364453 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:29.364656 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:29.364657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:29.364658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:29.364768 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:29.364946 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.364983 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.365071 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260927 11:10:29.365710 32507 heartbeater.cc:347] Connected to a master server at 127.17.95.254:36575
I20260927 11:10:29.365789 32507 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:29.365964 32507 heartbeater.cc:511] Master 127.17.95.254:36575 requested a full tablet report, sending...
I20260927 11:10:29.366324 32049 ts_manager.cc:194] Registered new tserver with Master: a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243)
I20260927 11:10:29.366670 32049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:39331
I20260927 11:10:29.377338 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:10:29.380254 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.377670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.17.95.254:36575)
0927 11:10:29.377750 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.377752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.377850 (+ 98us) server_negotiation.cc:407] Connection header received
0927 11:10:29.377939 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.377940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.377981 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.378078 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:10:29.378188 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.378652 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.379481 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.379706 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.379778 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.379934 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:10:29.380036 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:10:29.380136 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.380155 (+ 19us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.380177 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 11:10:29.380544 32049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:10:29.382031 32049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:10:29.384119 32171 tablet_service.cc:1467] Processing CreateTablet for tablet 3a181c3c4b0246c3bc0961598a23e0d5 (DEFAULT_TABLE table=test-table [id=f33c6bbbbbae4fad9eaf9cdbc1b538f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:29.384305 32171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a181c3c4b0246c3bc0961598a23e0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:29.385919 32129 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Bootstrap starting.
I20260927 11:10:29.386276 32129 tablet_bootstrap.cc:654] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:29.387593 32129 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: No bootstrap required, opened a new log
I20260927 11:10:29.387665 32129 ts_tablet_manager.cc:1397] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260927 11:10:29.387822 32129 raft_consensus.cc:348] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.387876 32129 raft_consensus.cc:374] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:29.387909 32129 raft_consensus.cc:729] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7468294f1c44411787753b1bdf7702e4, State: Initialized, Role: FOLLOWER
I20260927 11:10:29.387975 32129 consensus_queue.cc:260] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.388077 32129 ts_tablet_manager.cc:1428] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.388362 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.383813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35243 (local address 127.0.0.1:48218)
0927 11:10:29.383853 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.383864 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.383897 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.384575 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.384576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.384577 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:29.384629 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.384631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.385889 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.385890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.387167 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.387170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.388060 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.388061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.388137 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.388161 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.388239 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.388294 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 11:10:29.388386 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.384124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.17.95.195:35243)
0927 11:10:29.384372 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.384376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.384394 (+ 18us) server_negotiation.cc:407] Connection header received
0927 11:10:29.384432 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.384434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.384450 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.384535 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:29.384723 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.385748 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.387236 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.387947 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.388048 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.388111 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.388261 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.388307 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
I20260927 11:10:29.389320 32441 tablet_service.cc:1467] Processing CreateTablet for tablet 3a181c3c4b0246c3bc0961598a23e0d5 (DEFAULT_TABLE table=test-table [id=f33c6bbbbbae4fad9eaf9cdbc1b538f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:29.389673 32441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a181c3c4b0246c3bc0961598a23e0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:29.389845 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.383952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:39257 (local address 127.0.0.1:43870)
0927 11:10:29.384096 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.384107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.384162 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.386139 (+ 1977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.386140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.386142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:29.386183 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.386184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.387168 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.387169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.388543 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.388545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.389600 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.389601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.389661 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.389665 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.389716 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.389772 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260927 11:10:29.389876 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.384065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.17.95.194:39257)
0927 11:10:29.385962 (+ 1897us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.385966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.385976 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:10:29.386013 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.386016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.386031 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.386131 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:29.386235 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.387039 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.388634 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.389374 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.389451 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.389503 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.389754 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.389800 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1859,"thread_start_us":91,"threads_started":1}
I20260927 11:10:29.390461 32309 tablet_service.cc:1467] Processing CreateTablet for tablet 3a181c3c4b0246c3bc0961598a23e0d5 (DEFAULT_TABLE table=test-table [id=f33c6bbbbbae4fad9eaf9cdbc1b538f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:29.390765 32309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a181c3c4b0246c3bc0961598a23e0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:29.392617 32517 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Bootstrap starting.
I20260927 11:10:29.393170 32517 tablet_bootstrap.cc:654] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:29.393713 32518 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Bootstrap starting.
I20260927 11:10:29.393821 32517 log.cc:824] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:29.394337 32518 tablet_bootstrap.cc:654] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:29.394614 32517 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: No bootstrap required, opened a new log
I20260927 11:10:29.394716 32517 ts_tablet_manager.cc:1397] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:10:29.394852 32518 log.cc:824] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:29.395620 32518 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: No bootstrap required, opened a new log
I20260927 11:10:29.395685 32518 ts_tablet_manager.cc:1397] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:10:29.396658 32517 raft_consensus.cc:348] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.396816 32517 raft_consensus.cc:374] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:29.396852 32517 raft_consensus.cc:729] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a812cfd376594cbabadf775b852684c4, State: Initialized, Role: FOLLOWER
I20260927 11:10:29.396971 32517 consensus_queue.cc:260] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.397236 32517 ts_tablet_manager.cc:1428] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:10:29.397259 32507 heartbeater.cc:503] Master 127.17.95.254:36575 was elected leader, sending a full tablet report...
I20260927 11:10:29.397604 32518 raft_consensus.cc:348] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.397775 32518 raft_consensus.cc:374] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:29.397814 32518 raft_consensus.cc:729] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc8c97ca05347d49a9936b39ff62403, State: Initialized, Role: FOLLOWER
I20260927 11:10:29.397926 32518 consensus_queue.cc:260] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.398226 32518 ts_tablet_manager.cc:1428] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:10:29.398250 32375 heartbeater.cc:503] Master 127.17.95.254:36575 was elected leader, sending a full tablet report...
W20260927 11:10:29.486621 32376 tablet.cc:2369] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:10:29.546489 32242 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:29.546618 32242 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.546921 32242 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
I20260927 11:10:29.550040 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.547249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:60485 (local address 127.17.95.194:39257)
0927 11:10:29.547324 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.547326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.547597 (+ 271us) server_negotiation.cc:407] Connection header received
0927 11:10:29.547653 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.547654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.547664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.547736 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:29.547909 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.548369 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.549271 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.549663 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.549751 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.549760 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.549932 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.549967 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:10:29.550065 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.547321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:39257 (local address 127.17.95.193:60485)
0927 11:10:29.547490 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.547498 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.547557 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.547742 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.547743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.547744 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:29.547839 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.547840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.548455 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.548456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.549182 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.549183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.549776 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.549777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.549844 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.549864 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.549928 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.549965 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
I20260927 11:10:29.550150 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.547169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35243 (local address 127.17.95.193:55339)
0927 11:10:29.547240 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.547253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.547336 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.547526 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:29.547527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:29.547529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:29.547615 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.547617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.548345 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.548346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.549191 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:29.549193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.549846 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.549847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:29.549936 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.549940 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:10:29.550015 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 11:10:29.550083 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 11:10:29.550139 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.547241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:55339 (local address 127.17.95.195:35243)
0927 11:10:29.547335 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.547338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.547357 (+ 19us) server_negotiation.cc:407] Connection header received
0927 11:10:29.547422 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.547423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.547437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.547516 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:29.547719 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.548243 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.549301 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.549740 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.549833 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.549843 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.550022 (+ 179us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.550063 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 11:10:29.550580 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a181c3c4b0246c3bc0961598a23e0d5" candidate_uuid: "7468294f1c44411787753b1bdf7702e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc8c97ca05347d49a9936b39ff62403" is_pre_election: true
I20260927 11:10:29.550580 32461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a181c3c4b0246c3bc0961598a23e0d5" candidate_uuid: "7468294f1c44411787753b1bdf7702e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a812cfd376594cbabadf775b852684c4" is_pre_election: true
I20260927 11:10:29.550740 32329 raft_consensus.cc:2454] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7468294f1c44411787753b1bdf7702e4 in term 0.
I20260927 11:10:29.550760 32461 raft_consensus.cc:2454] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7468294f1c44411787753b1bdf7702e4 in term 0.
I20260927 11:10:29.551009 32124 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4, 7bc8c97ca05347d49a9936b39ff62403; no voters:
I20260927 11:10:29.551209 32242 raft_consensus.cc:2781] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:10:29.551277 32242 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:10:29.551311 32242 raft_consensus.cc:3037] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:29.551990 32242 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.552156 32242 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 1 election: Requested vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
I20260927 11:10:29.552295 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a181c3c4b0246c3bc0961598a23e0d5" candidate_uuid: "7468294f1c44411787753b1bdf7702e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc8c97ca05347d49a9936b39ff62403"
I20260927 11:10:29.552328 32461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a181c3c4b0246c3bc0961598a23e0d5" candidate_uuid: "7468294f1c44411787753b1bdf7702e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a812cfd376594cbabadf775b852684c4"
I20260927 11:10:29.552369 32329 raft_consensus.cc:3037] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:29.552397 32461 raft_consensus.cc:3037] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:29.553239 32329 raft_consensus.cc:2454] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7468294f1c44411787753b1bdf7702e4 in term 1.
I20260927 11:10:29.553239 32461 raft_consensus.cc:2454] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7468294f1c44411787753b1bdf7702e4 in term 1.
I20260927 11:10:29.553447 32123 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4, a812cfd376594cbabadf775b852684c4; no voters:
I20260927 11:10:29.553576 32242 raft_consensus.cc:2781] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:29.553757 32242 raft_consensus.cc:686] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7468294f1c44411787753b1bdf7702e4, State: Running, Role: LEADER
I20260927 11:10:29.553841 32242 consensus_queue.cc:237] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.554738 32049 catalog_manager.cc:5495] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7468294f1c44411787753b1bdf7702e4 (127.17.95.193). New cstate: current_term: 1 leader_uuid: "7468294f1c44411787753b1bdf7702e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } health_report { overall_health: HEALTHY } } }
I20260927 11:10:29.567837 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.565343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.17.95.193:38549)
0927 11:10:29.565404 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:29.565406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:29.565417 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:10:29.565447 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:29.565448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:29.565464 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:29.565556 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:10:29.565619 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.566128 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.566893 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:29.567175 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:29.567261 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:29.567436 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:10:29.567598 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:10:29.567665 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:29.567706 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 11:10:29.567726 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:10:29.569363 32461 raft_consensus.cc:1261] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Refusing update from remote peer 7468294f1c44411787753b1bdf7702e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:29.569363 32329 raft_consensus.cc:1261] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Refusing update from remote peer 7468294f1c44411787753b1bdf7702e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:29.569787 32242 consensus_queue.cc:1035] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:29.569927 32524 consensus_queue.cc:1035] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:29.572357 32527 mvcc.cc:205] Tried to move back new op lower bound from 7333918431513284608 to 7333918431453184000. Current Snapshot: MvccSnapshot[applied={T|T < 7333918431513284608}]
I20260927 11:10:29.572733 32529 mvcc.cc:205] Tried to move back new op lower bound from 7333918431513284608 to 7333918431453184000. Current Snapshot: MvccSnapshot[applied={T|T < 7333918431513284608}]
I20260927 11:10:29.574692 32049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:29.574782 32049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:29.575495 32049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Sending DeleteTablet for 3 replicas of tablet 3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:29.575817 32441 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a181c3c4b0246c3bc0961598a23e0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:29 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:48218
I20260927 11:10:29.575874 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 32088
I20260927 11:10:29.575914 32309 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a181c3c4b0246c3bc0961598a23e0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:29 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43870
I20260927 11:10:29.576228 32537 tablet_replica.cc:330] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: stopping tablet replica
I20260927 11:10:29.576363 32537 raft_consensus.cc:2229] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:29.576437 32538 tablet_replica.cc:330] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: stopping tablet replica
I20260927 11:10:29.576467 32537 raft_consensus.cc:2258] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:29.576529 32538 raft_consensus.cc:2229] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:29.576685 32538 raft_consensus.cc:2258] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:29.576856 32537 ts_tablet_manager.cc:1905] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:29.577090 32538 ts_tablet_manager.cc:1905] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:29.578361 32537 ts_tablet_manager.cc:1918] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:29.578445 32537 log.cc:1196] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:29.578718 32537 ts_tablet_manager.cc:1939] T 3a181c3c4b0246c3bc0961598a23e0d5 P a812cfd376594cbabadf775b852684c4: Deleting consensus metadata
I20260927 11:10:29.579044 32538 ts_tablet_manager.cc:1918] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:29.579154 32538 log.cc:1196] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:29.579115 32033 catalog_manager.cc:4841] TS a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): tablet 3a181c3c4b0246c3bc0961598a23e0d5 (table test-table [id=f33c6bbbbbae4fad9eaf9cdbc1b538f6]) successfully deleted
I20260927 11:10:29.579469 32538 ts_tablet_manager.cc:1939] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7bc8c97ca05347d49a9936b39ff62403: Deleting consensus metadata
I20260927 11:10:29.579829 32034 catalog_manager.cc:4841] TS 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): tablet 3a181c3c4b0246c3bc0961598a23e0d5 (table test-table [id=f33c6bbbbbae4fad9eaf9cdbc1b538f6]) successfully deleted
W20260927 11:10:29.582532 32035 connection.cc:535] client connection to 127.17.95.193:38549 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 11:10:29.582639 32035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:10:29.583113 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.582946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:38549 (local address 127.0.0.1:40956)
0927 11:10:29.582996 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.583030 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 11:10:29.583165 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 32243
W20260927 11:10:29.583238 32035 catalog_manager.cc:4584] TS 7468294f1c44411787753b1bdf7702e4 (127.17.95.193:38549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a181c3c4b0246c3bc0961598a23e0d5: Network error: Client connection negotiation failed: client connection to 127.17.95.193:38549: connect: Connection refused (error 111)
I20260927 11:10:29.588559 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 32378
I20260927 11:10:29.593034 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 32016
I20260927 11:10:29.598042 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36575
--webserver_interface=127.17.95.254
--webserver_port=33839
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:36575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:29.675755 32540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:29.675923 32540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:29.675942 32540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:29.676957 32540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:29.676987 32540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:29.677000 32540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:29.677012 32540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:29.677023 32540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:29.678381 32540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:36575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=33839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:29.678553 32540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:29.678709 32540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:29.683064 32546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.683219 32548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.683079 32545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:29.683534 32540 server_base.cc:1034] running on GCE node
I20260927 11:10:29.683743 32540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:29.683990 32540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:29.685128 32540 hybrid_clock.cc:648] HybridClock initialized: now 1790507429685117 us; error 25 us; skew 500 ppm
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:29.687623 32540 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:29.688060 32540 webserver.cc:460] Webserver started at http://127.17.95.254:33839/ using document root <none> and password file <none>
I20260927 11:10:29.688287 32540 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:29.688331 32540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:29.689584 32540 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:29.690241 32555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.690418 32540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.690497 32540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "de66082f6b8a466991d05c696dded292"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "067f031f4a93c9895977d2308978d8d4"
server_key_iv: "130996815676b5740d02c4c8229cb911"
server_key_version: "encryptionkey@0"
I20260927 11:10:29.690785 32540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:29.706451 32540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:29.708365 32540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:29.708469 32540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:29.712019 32540 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:36575
I20260927 11:10:29.712076 32607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:36575 every 8 connection(s)
I20260927 11:10:29.712445 32540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:29.713855 32608 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:10:29.714447 32608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap starting.
I20260927 11:10:29.722292 32608 log.cc:824] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:29.722333 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 32540
I20260927 11:10:29.722620 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:38549
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:36575
--webserver_port=36423
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:10:29.724494 32608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:10:29.724781 32608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: Bootstrap complete.
I20260927 11:10:29.726938 32608 raft_consensus.cc:348] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:29.727217 32608 raft_consensus.cc:729] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: de66082f6b8a466991d05c696dded292, State: Initialized, Role: FOLLOWER
I20260927 11:10:29.727444 32608 consensus_queue.cc:260] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:29.727535 32608 raft_consensus.cc:388] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:29.727588 32608 raft_consensus.cc:482] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:29.727643 32608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 2 FOLLOWER]: Advancing to term 3
I20260927 11:10:29.728560 32608 raft_consensus.cc:504] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:29.728678 32608 leader_election.cc:302] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de66082f6b8a466991d05c696dded292; no voters:
I20260927 11:10:29.728892 32608 leader_election.cc:288] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 11:10:29.728938 32612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 11:10:29.729070 32612 raft_consensus.cc:686] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [term 3 LEADER]: Becoming Leader. State: Replica: de66082f6b8a466991d05c696dded292, State: Running, Role: LEADER
I20260927 11:10:29.729174 32608 sys_catalog.cc:564] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:29.729212 32612 consensus_queue.cc:237] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } }
I20260927 11:10:29.729578 32614 sys_catalog.cc:455] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "de66082f6b8a466991d05c696dded292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } } }
I20260927 11:10:29.729676 32614 sys_catalog.cc:458] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:29.729833 32613 sys_catalog.cc:455] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de66082f6b8a466991d05c696dded292. Latest consensus state: current_term: 3 leader_uuid: "de66082f6b8a466991d05c696dded292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de66082f6b8a466991d05c696dded292" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36575 } } }
I20260927 11:10:29.729923 32613 sys_catalog.cc:458] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:29.730844 32628 catalog_manager.cc:1259] Loaded cluster ID: fb8a04c6321844e9918344d7f06b3ada
I20260927 11:10:29.730890 32628 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: loading cluster ID for follower catalog manager: success
W20260927 11:10:29.731102 32628 catalog_manager.cc:1576] T 00000000000000000000000000000000 P de66082f6b8a466991d05c696dded292: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260927 11:10:29.731148 32628 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260927 11:10:29.731204 32621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:29.731564 32621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:29.731622 32621 catalog_manager.cc:1259] Loaded cluster ID: fb8a04c6321844e9918344d7f06b3ada
I20260927 11:10:29.731652 32621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 11:10:29.731786 32621 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2c51aaa9ad google::LogMessage::Fail() at ??:0
@ 0x7f2c51aad88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2c51aaa4de google::LogMessage::Flush() at ??:0
@ 0x7f2c51aaba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2c599cb0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2c599d48b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2c539780d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2c5396d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2c547d16db start_thread at ??:0
@ 0x7f2c505a861f clone at ??:0
W20260927 11:10:29.814536 32611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:29.814711 32611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:29.814733 32611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:29.815881 32611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:29.815920 32611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:29.815944 32611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:10:29.817286 32611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:38549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=36423
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:29.817505 32611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:29.817739 32611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:29.822853 32645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.822834 32647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.822870 32644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:29.823397 32611 server_base.cc:1034] running on GCE node
I20260927 11:10:29.823585 32611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:29.823827 32611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:29.824985 32611 hybrid_clock.cc:648] HybridClock initialized: now 1790507429824971 us; error 34 us; skew 500 ppm
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:29.829171 32611 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:10:29.829799 32611 webserver.cc:460] Webserver started at http://127.17.95.193:36423/ using document root <none> and password file <none>
I20260927 11:10:29.830250 32611 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:29.830336 32611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:29.832250 32611 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:29.833014 32654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.833250 32611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:29.833334 32611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7468294f1c44411787753b1bdf7702e4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "68c5964b5230d340e24b39577f11c6af"
server_key_iv: "0e77d090091e7777167b791b31faa21e"
server_key_version: "encryptionkey@0"
I20260927 11:10:29.833765 32611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:29.865924 32611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:29.868582 32611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:29.868727 32611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:29.869038 32611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:29.869529 32661 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 11:10:29.870941 32611 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 11:10:29.871019 32611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 11:10:29.871121 32611 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 11:10:29.872231 32611 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 11:10:29.872301 32611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:29.872334 32661 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Bootstrap starting.
I20260927 11:10:29.879974 32611 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:38549
I20260927 11:10:29.880036 300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:38549 every 8 connection(s)
I20260927 11:10:29.880446 32611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:10:29.882939 32661 log.cc:824] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:29.884904 32661 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:10:29.885175 32661 tablet_bootstrap.cc:492] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Bootstrap complete.
I20260927 11:10:29.886018 32661 ts_tablet_manager.cc:1397] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20260927 11:10:29.886934 32661 raft_consensus.cc:348] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.887225 32661 raft_consensus.cc:729] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7468294f1c44411787753b1bdf7702e4, State: Initialized, Role: FOLLOWER
I20260927 11:10:29.887393 32661 consensus_queue.cc:260] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:29.887689 32661 ts_tablet_manager.cc:1428] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260927 11:10:29.887972 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 32611
I20260927 11:10:29.888222 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:39257
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:36575
--webserver_port=38247
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:29.947463 303 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:10:29.881054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:41733)
0927 11:10:29.881216 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.881255 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:29.881988 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:29.947231 (+ 65243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
W20260927 11:10:29.947786 301 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:36575 (0 consecutive failures): Network error: Failed to ping master at 127.17.95.254:36575: Client connection negotiation failed: client connection to 127.17.95.254:36575: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:10:29.948177 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.948052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:59719)
0927 11:10:29.948106 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.948153 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 11:10:29.948566 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:29.948456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:35829)
0927 11:10:29.948512 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:29.948546 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 11:10:29.948685 301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:10:29.972991 306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:29.973187 306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:29.973210 306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:29.974279 306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:29.974313 306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:29.974339 306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:10:29.975798 306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:39257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=38247
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:29.975968 306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:29.976140 306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:29.981051 312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.981087 311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:29.981156 314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:29.981292 306 server_base.cc:1034] running on GCE node
I20260927 11:10:29.981520 306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:29.981791 306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:29.982959 306 hybrid_clock.cc:648] HybridClock initialized: now 1790507429982940 us; error 30 us; skew 500 ppm
Sep 27 11:10:29 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:29.985460 306 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:10:29.985950 306 webserver.cc:460] Webserver started at http://127.17.95.194:38247/ using document root <none> and password file <none>
I20260927 11:10:29.986198 306 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:29.986266 306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:29.987637 306 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 11:10:29.988260 321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:29.988437 306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:29.988533 306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "7bc8c97ca05347d49a9936b39ff62403"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "2a5bc9d52a8cbd8315fc8873949b178a"
server_key_iv: "c85e693f81b6af505d374f5f777ce3ff"
server_key_version: "encryptionkey@0"
I20260927 11:10:29.988871 306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:30.006371 306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:30.008834 306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:30.009004 306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:30.009243 306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:30.009565 306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:30.009624 306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:30.009668 306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:30.009701 306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:30.016104 306 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:39257
I20260927 11:10:30.016140 434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:39257 every 8 connection(s)
I20260927 11:10:30.016484 306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:10:30.017544 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.017170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:35627)
0927 11:10:30.017317 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.017416 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
W20260927 11:10:30.017932 435 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:36575 (0 consecutive failures): Network error: Failed to ping master at 127.17.95.254:36575: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
I20260927 11:10:30.018195 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.018104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:49747)
0927 11:10:30.018152 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.018178 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 11:10:30.018462 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.018391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:48169)
0927 11:10:30.018430 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.018450 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260927 11:10:30.018625 435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 11:10:30.023542 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 306
I20260927 11:10:30.023797 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:35243
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:36575
--webserver_port=39487
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:30.105347 438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:30.105538 438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:30.105563 438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:30.106593 438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:30.106630 438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:30.106657 438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:10:30.108024 438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:35243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=39487
--tserver_master_addrs=127.17.95.254:36575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:30.108206 438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:30.108371 438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:30.113237 444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:30.113467 438 server_base.cc:1034] running on GCE node
W20260927 11:10:30.113247 443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:30.113247 446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:30.113811 438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:30.114040 438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:30.115140 438 hybrid_clock.cc:648] HybridClock initialized: now 1790507430115128 us; error 27 us; skew 500 ppm
Sep 27 11:10:30 dist-test-slave-6j0v krb5kdc[31471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507430, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:30.117472 438 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:10:30.117924 438 webserver.cc:460] Webserver started at http://127.17.95.195:39487/ using document root <none> and password file <none>
I20260927 11:10:30.118137 438 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:30.118203 438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:30.119508 438 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:30.120064 453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:30.120255 438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:30.120350 438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "a812cfd376594cbabadf775b852684c4"
format_stamp: "Formatted at 2026-09-27 11:10:28 on dist-test-slave-6j0v"
server_key: "1add9e9892b96c945050b65b09c013f5"
server_key_iv: "a8748b036a7d66140399b061f648769e"
server_key_version: "encryptionkey@0"
I20260927 11:10:30.120646 438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:30.138914 438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:30.140724 438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:30.140865 438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:30.141069 438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:30.141353 438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:30.141402 438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:30.141441 438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:30.141467 438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:30.147028 438 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35243
I20260927 11:10:30.147084 566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35243 every 8 connection(s)
I20260927 11:10:30.147464 438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:10:30.148283 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 438
I20260927 11:10:30.148518 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.148078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:46461)
0927 11:10:30.148293 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.148365 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":84,"threads_started":1}
W20260927 11:10:30.148921 567 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:36575 (0 consecutive failures): Network error: Failed to ping master at 127.17.95.254:36575: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
I20260927 11:10:30.149209 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.149103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:50009)
0927 11:10:30.149157 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.149187 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 11:10:30.149547 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.149480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:39467)
0927 11:10:30.149517 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.149536 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 11:10:30.149676 567 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 11:10:30.949236 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:30.949011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:43845)
0927 11:10:30.949141 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:30.949176 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260927 11:10:31.019199 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.018918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:42897)
0927 11:10:31.019052 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:31.019151 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260927 11:10:31.150144 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.149954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:43449)
0927 11:10:31.150063 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:31.150092 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":51,"threads_started":1}
I20260927 11:10:31.513870 574 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:31.513980 574 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:31.514402 574 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
Sep 27 11:10:31 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.194@KRBTEST.COM
Sep 27 11:10:31 dist-test-slave-6j0v krb5kdc[31471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507429, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.195@KRBTEST.COM
I20260927 11:10:31.524974 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.514648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:45577 (local address 127.17.95.195:35243)
0927 11:10:31.514760 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:31.514778 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:31.514828 (+ 50us) server_negotiation.cc:407] Connection header received
0927 11:10:31.515367 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:31.515370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:31.515414 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:31.515564 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:31.516257 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.517034 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.517943 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.518337 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.518450 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:31.521213 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:31.521216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:31.521217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:31.521226 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:31.522948 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523426 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:31.523429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:31.523429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:31.523526 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523899 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:31.523900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:31.523900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:31.524257 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:31.524348 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:31.524726 (+ 378us) server_negotiation.cc:299] Negotiation successful
0927 11:10:31.524866 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":51,"threads_started":1}
I20260927 11:10:31.524974 578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.514855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:42355 (local address 127.17.95.194:39257)
0927 11:10:31.514991 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:31.514999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:31.515027 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:10:31.515487 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:31.515489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:31.515526 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:31.515667 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:31.516245 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.517044 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.517948 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.518332 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.518450 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:31.520835 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:31.520838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:31.520839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:31.520856 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:31.522707 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523373 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:31.523376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:31.523377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:31.523487 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523899 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:31.523900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:31.523900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:31.524253 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:31.524345 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:31.524726 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 11:10:31.524863 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
I20260927 11:10:31.524999 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.514552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35243 (local address 127.17.95.193:45577)
0927 11:10:31.514698 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:31.514709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:31.514769 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:31.515601 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:31.515604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:31.515940 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:31.516133 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:31.516139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.517137 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.517138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:31.517862 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:31.517864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.518427 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.518428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:31.518542 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:31.519237 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:31.519245 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:31.521096 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:31.523051 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:31.523052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:31.523344 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:31.523607 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:31.523608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:31.523792 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:31.524343 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:31.524348 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:31.524513 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 11:10:31.524635 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 11:10:31.524860 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":73,"thread_start_us":81,"threads_started":1}
I20260927 11:10:31.525053 577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.514662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:39257 (local address 127.17.95.193:42355)
0927 11:10:31.514838 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:31.514846 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:31.514886 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:31.515654 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:31.515655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:31.515942 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:31.516133 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:31.516139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.517152 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.517153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:31.517851 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:31.517852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:31.518420 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:31.518421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:31.518542 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:31.519290 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:31.519294 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:31.520713 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:31.522818 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:31.522825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:31.522827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:31.523277 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:31.523604 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:31.523605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:31.523605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:31.523792 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:31.524341 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:31.524348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:31.524513 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 11:10:31.524645 (+ 132us) client_negotiation.cc:240] Negotiation successful
0927 11:10:31.524919 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":139,"thread_start_us":45,"threads_started":1}
W20260927 11:10:31.526063 32655 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
W20260927 11:10:31.526217 32656 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:31.526283 32656 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4; no voters: 7bc8c97ca05347d49a9936b39ff62403, a812cfd376594cbabadf775b852684c4
I20260927 11:10:31.526404 574 raft_consensus.cc:2726] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:10:31.949828 577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:31.949690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:55733)
0927 11:10:31.949746 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:31.949787 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 11:10:32.019826 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:32.019612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:53239)
0927 11:10:32.019741 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:32.019778 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260927 11:10:32.150768 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:32.150575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:52095)
0927 11:10:32.150697 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:32.150732 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
I20260927 11:10:32.950584 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:32.950317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:50529)
0927 11:10:32.950496 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:32.950533 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260927 11:10:33.020498 582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:33.020280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:48939)
0927 11:10:33.020411 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:33.020447 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260927 11:10:33.151417 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:33.151207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:38483)
0927 11:10:33.151343 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:33.151375 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260927 11:10:33.372287 584 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:33.372391 584 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:33.372551 584 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
W20260927 11:10:33.372918 32655 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
W20260927 11:10:33.372999 32656 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:33.373114 32656 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4; no voters: 7bc8c97ca05347d49a9936b39ff62403, a812cfd376594cbabadf775b852684c4
I20260927 11:10:33.373195 584 raft_consensus.cc:2726] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:10:33.951422 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:33.951196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:36143)
0927 11:10:33.951337 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:33.951374 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260927 11:10:34.021183 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:34.020953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:48515)
0927 11:10:34.021100 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:34.021133 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":57,"threads_started":1}
I20260927 11:10:34.152138 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:34.151931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:59483)
0927 11:10:34.152059 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:34.152100 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260927 11:10:34.952160 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:34.951904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:42353)
0927 11:10:34.952063 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:34.952103 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260927 11:10:34.958604 589 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:34.958691 589 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:34.958900 589 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
W20260927 11:10:34.959357 32656 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
W20260927 11:10:34.959445 32655 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:34.959509 32655 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4; no voters: 7bc8c97ca05347d49a9936b39ff62403, a812cfd376594cbabadf775b852684c4
I20260927 11:10:34.959604 589 raft_consensus.cc:2726] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:10:35.021792 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:35.021585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:43793)
0927 11:10:35.021721 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:35.021754 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260927 11:10:35.152844 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:35.152620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:40403)
0927 11:10:35.152746 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:35.152794 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260927 11:10:35.952721 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:35.952522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:38907)
0927 11:10:35.952636 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:35.952668 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260927 11:10:36.022475 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:36.022280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:35867)
0927 11:10:36.022405 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:36.022439 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260927 11:10:36.153481 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:36.153284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:47319)
0927 11:10:36.153418 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:36.153446 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
I20260927 11:10:36.953315 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:36.953105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:52089)
0927 11:10:36.953235 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:36.953268 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260927 11:10:37.023195 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:37.022909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:46969)
0927 11:10:37.023111 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:37.023148 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260927 11:10:37.154057 597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:37.153842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:37219)
0927 11:10:37.153991 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:37.154022 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260927 11:10:37.944820 598 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:37.944913 598 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:37.945137 598 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
W20260927 11:10:37.945573 32656 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
W20260927 11:10:37.945658 32655 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:37.945752 32655 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4; no voters: 7bc8c97ca05347d49a9936b39ff62403, a812cfd376594cbabadf775b852684c4
I20260927 11:10:37.945837 598 raft_consensus.cc:2726] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:10:37.953922 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:37.953736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:36875)
0927 11:10:37.953852 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:37.953885 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
I20260927 11:10:38.023784 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:38.023558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:55507)
0927 11:10:38.023708 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:38.023743 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260927 11:10:38.154737 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:38.154531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:39809)
0927 11:10:38.154665 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:38.154698 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260927 11:10:38.954659 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:38.954387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:48059)
0927 11:10:38.954563 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:38.954607 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":60,"threads_started":1}
I20260927 11:10:39.024384 603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:39.024198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:47577)
0927 11:10:39.024314 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:39.024346 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260927 11:10:39.155418 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:39.155223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:37781)
0927 11:10:39.155350 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:39.155383 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260927 11:10:39.955427 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:39.955228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:45579)
0927 11:10:39.955339 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:39.955380 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":51,"threads_started":1}
I20260927 11:10:40.025045 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:40.024865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:47467)
0927 11:10:40.024974 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:40.025007 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260927 11:10:40.156030 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:40.155838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:39783)
0927 11:10:40.155959 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:40.155992 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260927 11:10:40.956153 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:40.955901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:60769)
0927 11:10:40.956047 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:40.956085 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260927 11:10:41.025743 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:41.025536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:47153)
0927 11:10:41.025671 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:41.025703 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260927 11:10:41.156633 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:41.156436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:34043)
0927 11:10:41.156548 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:41.156581 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260927 11:10:41.366896 611 raft_consensus.cc:482] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:41.366989 611 raft_consensus.cc:504] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a812cfd376594cbabadf775b852684c4" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35243 } } peers { permanent_uuid: "7bc8c97ca05347d49a9936b39ff62403" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 39257 } } peers { permanent_uuid: "7468294f1c44411787753b1bdf7702e4" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 38549 } }
I20260927 11:10:41.367321 611 leader_election.cc:288] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243), 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257)
W20260927 11:10:41.367612 32655 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a812cfd376594cbabadf775b852684c4 (127.17.95.195:35243): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
W20260927 11:10:41.367681 32656 leader_election.cc:341] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bc8c97ca05347d49a9936b39ff62403 (127.17.95.194:39257): Not found: Tablet not found: 3a181c3c4b0246c3bc0961598a23e0d5
I20260927 11:10:41.367761 32656 leader_election.cc:302] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7468294f1c44411787753b1bdf7702e4; no voters: 7bc8c97ca05347d49a9936b39ff62403, a812cfd376594cbabadf775b852684c4
I20260927 11:10:41.367838 611 raft_consensus.cc:2726] T 3a181c3c4b0246c3bc0961598a23e0d5 P 7468294f1c44411787753b1bdf7702e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:10:41.956959 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:41.956722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:37911)
0927 11:10:41.956859 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:41.956896 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260927 11:10:42.026363 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:42.026165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:58961)
0927 11:10:42.026279 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:42.026311 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260927 11:10:42.157294 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:42.157092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:58835)
0927 11:10:42.157221 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:42.157254 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260927 11:10:42.957718 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:42.957503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:45499)
0927 11:10:42.957636 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:42.957672 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260927 11:10:43.026989 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:43.026810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:41081)
0927 11:10:43.026920 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:43.026952 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260927 11:10:43.157972 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:43.157769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:57473)
0927 11:10:43.157893 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:43.157938 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260927 11:10:43.958453 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:43.958254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:56939)
0927 11:10:43.958372 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:43.958409 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260927 11:10:44.027650 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:44.027423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:54873)
0927 11:10:44.027569 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:44.027605 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260927 11:10:44.158615 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:44.158426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.195:47165)
0927 11:10:44.158538 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:44.158575 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260927 11:10:44.959250 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:44.958986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.193:52271)
0927 11:10:44.959152 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:44.959187 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":53,"threads_started":1}
I20260927 11:10:45.028406 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:45.028160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:36575 (local address 127.17.95.194:33849)
0927 11:10:45.028317 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:45.028361 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:36575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260927 11:10:45.149782 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 32611
I20260927 11:10:45.154493 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 306
I20260927 11:10:45.158672 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 438
2026-09-27T11:10:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18115 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:10:45 dist-test-slave-6j0v krb5kdc[625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:10:45 dist-test-slave-6j0v krb5kdc[625](info): set up 2 sockets
Sep 27 11:10:45 dist-test-slave-6j0v krb5kdc[625](info): commencing operation
krb5kdc: starting...
W20260927 11:10:46.187216 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:10:46.298586 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44343
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:44343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:46.376065 638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:46.376259 638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:46.376281 638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:46.377336 638 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 11:10:46.377372 638 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 11:10:46.377386 638 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 11:10:46.377400 638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:10:46.377414 638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:46.377429 638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:46.377442 638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:10:46.377457 638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:10:46.378904 638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:44343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:46.379168 638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:46.379431 638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:46.384126 646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:46.384146 644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:46.384220 638 server_base.cc:1034] running on GCE node
W20260927 11:10:46.384126 643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:46.384590 638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:46.384851 638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:46.386008 638 hybrid_clock.cc:648] HybridClock initialized: now 1790507446385987 us; error 41 us; skew 500 ppm
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:46.388548 638 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:10:46.389025 638 webserver.cc:460] Webserver started at http://127.17.95.254:39261/ using document root <none> and password file <none>
I20260927 11:10:46.389253 638 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:46.389320 638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:46.389469 638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:46.390626 638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "1e7eaebc174c4d2cb90baa51381f01f9"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "649164aa2400c4ff4e1a6206c9eff6b7"
server_key_iv: "002d5fbb0e2bc920d3459b7a00908370"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.390990 638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "1e7eaebc174c4d2cb90baa51381f01f9"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "649164aa2400c4ff4e1a6206c9eff6b7"
server_key_iv: "002d5fbb0e2bc920d3459b7a00908370"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.392403 638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:10:46.393193 653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.393395 638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:10:46.393502 638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "1e7eaebc174c4d2cb90baa51381f01f9"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "649164aa2400c4ff4e1a6206c9eff6b7"
server_key_iv: "002d5fbb0e2bc920d3459b7a00908370"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.393599 638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:46.408178 638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:46.410249 638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:46.410409 638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:46.414480 638 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:44343
I20260927 11:10:46.414515 705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:44343 every 8 connection(s)
I20260927 11:10:46.414844 638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:10:46.415648 706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:46.417755 706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap starting.
I20260927 11:10:46.418267 706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:46.418488 706 log.cc:824] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:46.419059 706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: No bootstrap required, opened a new log
I20260927 11:10:46.420603 706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:10:46.420730 706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:46.420753 706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7eaebc174c4d2cb90baa51381f01f9, State: Initialized, Role: FOLLOWER
I20260927 11:10:46.420845 706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:10:46.420893 706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:10:46.420940 706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:10:46.420989 706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:46.421506 706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:10:46.421617 706 leader_election.cc:302] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e7eaebc174c4d2cb90baa51381f01f9; no voters:
I20260927 11:10:46.421782 706 leader_election.cc:288] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:10:46.421850 709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:46.421996 709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 LEADER]: Becoming Leader. State: Replica: 1e7eaebc174c4d2cb90baa51381f01f9, State: Running, Role: LEADER
I20260927 11:10:46.422025 706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:10:46.422098 709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:10:46.422693 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 638
I20260927 11:10:46.422785 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:10:46.422873 710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } } }
I20260927 11:10:46.422971 710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:46.423008 17791 external_mini_cluster.cc:1428] Setting key 4ebb4e800e2aeed56430482ce3c5dc9d
I20260927 11:10:46.423202 711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e7eaebc174c4d2cb90baa51381f01f9. Latest consensus state: current_term: 1 leader_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } } }
I20260927 11:10:46.423307 711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:10:46.423249 718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:10:46.423892 718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:10:46.425529 718 catalog_manager.cc:1347] Generated new cluster ID: 6546bbf76bc34293a358473b5c24373d
I20260927 11:10:46.425602 718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:46.433957 718 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:10:46.434428 718 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:10:46.435922 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.423682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.17.95.254:44343)
0927 11:10:46.424033 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.424042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.424084 (+ 42us) server_negotiation.cc:407] Connection header received
0927 11:10:46.424201 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.424208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.424265 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.424410 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:46.425183 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.426524 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.427686 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.428304 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.428378 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.431306 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:46.431310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:46.431310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:46.431322 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:46.433978 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.434368 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.434371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.434371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.434478 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.434682 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.434683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.434683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.434945 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:46.435065 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.435525 (+ 460us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.435700 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":61,"threads_started":1}
I20260927 11:10:46.439234 718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Generated new TSK 0
I20260927 11:10:46.439388 718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:10:46.469519 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44343
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:46.551784 732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:46.551990 732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:46.552026 732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:46.553134 732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:46.553215 732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:46.553259 732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:10:46.554782 732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:46.555029 732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:46.555258 732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:46.559890 738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:46.559958 737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:46.559912 740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:46.560113 732 server_base.cc:1034] running on GCE node
I20260927 11:10:46.560297 732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:46.560525 732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:46.561683 732 hybrid_clock.cc:648] HybridClock initialized: now 1790507446561658 us; error 46 us; skew 500 ppm
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:46.564226 732 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:10:46.564684 732 webserver.cc:460] Webserver started at http://127.17.95.193:33659/ using document root <none> and password file <none>
I20260927 11:10:46.564847 732 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:46.564884 732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:46.564965 732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:46.565850 732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "7ae020b2cd044833a31c53b3ec5ad84a"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "964aefa2b2183b32568a9dfa35c3376d"
server_key_iv: "61ce5c898fe856025bea5dc9041369c7"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.566121 732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "7ae020b2cd044833a31c53b3ec5ad84a"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "964aefa2b2183b32568a9dfa35c3376d"
server_key_iv: "61ce5c898fe856025bea5dc9041369c7"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.567322 732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:10:46.568003 747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.568197 732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:46.568286 732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7ae020b2cd044833a31c53b3ec5ad84a"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "964aefa2b2183b32568a9dfa35c3376d"
server_key_iv: "61ce5c898fe856025bea5dc9041369c7"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.568351 732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:46.577457 732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:46.579330 732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:46.579444 732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:46.579629 732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:46.579866 732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:46.579897 732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.579921 732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:46.579934 732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.585708 732 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41637
I20260927 11:10:46.585776 860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41637 every 8 connection(s)
I20260927 11:10:46.586076 732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:46.594036 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 732
I20260927 11:10:46.594146 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:10:46.594337 17791 external_mini_cluster.cc:1428] Setting key bc60c588983211187ca0b7d01fe91d47
I20260927 11:10:46.594951 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.586789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:43567 (local address 127.17.95.254:44343)
0927 11:10:46.586875 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.586877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.587000 (+ 123us) server_negotiation.cc:407] Connection header received
0927 11:10:46.587490 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.587491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.587500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.587555 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:46.588435 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.588934 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.589916 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.590204 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.590291 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.592249 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:46.592251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:46.592251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:46.592258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:46.593380 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.593794 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.593796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.593796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.593917 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.594210 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.594211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.594212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.594338 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:46.594435 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.594786 (+ 351us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.594892 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 11:10:46.595135 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.586713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:43567)
0927 11:10:46.586879 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.586916 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.587410 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.587625 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.587628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.588159 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:46.588334 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.588338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.589018 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.589020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.589823 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.589825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.590300 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.590301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.590450 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.590834 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:46.590841 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:46.592155 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:46.593469 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:46.593475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:46.593476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:46.593708 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:46.593990 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:46.593991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:46.593992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:46.594127 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:46.594432 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:46.594438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:46.594618 (+ 180us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.594720 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.594891 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260927 11:10:46.595643 861 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44343
I20260927 11:10:46.595733 861 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:46.595882 861 heartbeater.cc:511] Master 127.17.95.254:44343 requested a full tablet report, sending...
I20260927 11:10:46.596362 670 ts_manager.cc:194] Registered new tserver with Master: 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637)
I20260927 11:10:46.596938 670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:43567
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:10:46.618216 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44343
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:46.702554 866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:46.702772 866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:46.702808 866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:46.704005 866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:46.704063 866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:46.704107 866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:10:46.705505 866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:46.705749 866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:46.705955 866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:46.710789 871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:46.710831 866 server_base.cc:1034] running on GCE node
W20260927 11:10:46.710799 872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:46.710799 874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:46.711167 866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:46.711416 866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:46.712553 866 hybrid_clock.cc:648] HybridClock initialized: now 1790507446712548 us; error 33 us; skew 500 ppm
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:46.714916 866 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:10:46.715387 866 webserver.cc:460] Webserver started at http://127.17.95.194:40691/ using document root <none> and password file <none>
I20260927 11:10:46.715624 866 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:46.715667 866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:46.715809 866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:46.716804 866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "7d8c122f21662276d56db0ad9e1ac7cc"
server_key_iv: "4603732d6a209b1301e6cf5916b93f69"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.717156 866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "7d8c122f21662276d56db0ad9e1ac7cc"
server_key_iv: "4603732d6a209b1301e6cf5916b93f69"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.718412 866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:10:46.719206 881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.719394 866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:10:46.719480 866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "7d8c122f21662276d56db0ad9e1ac7cc"
server_key_iv: "4603732d6a209b1301e6cf5916b93f69"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.719566 866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:46.730577 866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:46.732452 866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:46.732607 866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:46.732831 866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:46.733116 866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:46.733167 866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.733206 866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:46.733232 866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.738787 866 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:36009
I20260927 11:10:46.738827 994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:36009 every 8 connection(s)
I20260927 11:10:46.739151 866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:10:46.744303 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 866
I20260927 11:10:46.744414 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:10:46.744639 17791 external_mini_cluster.cc:1428] Setting key 57a638050b4c085cff479a87b430ede6
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:46.748386 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.739812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:45589 (local address 127.17.95.254:44343)
0927 11:10:46.739852 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.739854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.740062 (+ 208us) server_negotiation.cc:407] Connection header received
0927 11:10:46.740533 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.740534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.740545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.740622 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:46.741446 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.741933 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.742902 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.743149 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.743199 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.745283 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:46.745284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:46.745284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:46.745291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:46.746852 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.747436 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.747437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.747437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.747497 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.747774 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.747775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.747775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.747873 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:46.747923 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.748262 (+ 339us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.748325 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:10:46.748742 997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.739767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:45589)
0927 11:10:46.739935 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.739979 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.740443 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.740675 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.740678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.741189 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:46.741361 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.741366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.742016 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.742017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.742823 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.742825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.743222 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.743223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.743317 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.743696 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:46.743703 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:46.745186 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:46.746967 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:46.746975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:46.746977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:46.747345 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:46.747570 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:46.747571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:46.747571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:46.747708 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:46.747934 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:46.747940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:46.748104 (+ 164us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.748222 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.748405 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260927 11:10:46.749209 995 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44343
I20260927 11:10:46.749305 995 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:46.749497 995 heartbeater.cc:511] Master 127.17.95.254:44343 requested a full tablet report, sending...
I20260927 11:10:46.749868 670 ts_manager.cc:194] Registered new tserver with Master: 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
I20260927 11:10:46.750265 670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:45589
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:10:46.769431 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44343
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:10:46.847815 1000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:10:46.847994 1000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:10:46.848017 1000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:10:46.849145 1000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:10:46.849189 1000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:10:46.849215 1000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:10:46.850636 1000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:10:46.850857 1000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:10:46.851094 1000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:10:46.855592 1006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:46.855581 1008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:10:46.855600 1005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:10:46.855966 1000 server_base.cc:1034] running on GCE node
I20260927 11:10:46.856160 1000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:10:46.856386 1000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:10:46.857563 1000 hybrid_clock.cc:648] HybridClock initialized: now 1790507446857538 us; error 33 us; skew 500 ppm
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:10:46.860148 1000 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:10:46.860531 1000 webserver.cc:460] Webserver started at http://127.17.95.195:35441/ using document root <none> and password file <none>
I20260927 11:10:46.860706 1000 fs_manager.cc:359] Metadata directory not provided
I20260927 11:10:46.860749 1000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:10:46.860833 1000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:10:46.861742 1000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "0774f62eea22d757dc11681c459ecb83"
server_key_iv: "5114b7752327ee65b83e04cbd62785b5"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.862033 1000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "0774f62eea22d757dc11681c459ecb83"
server_key_iv: "5114b7752327ee65b83e04cbd62785b5"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.863374 1000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:10:46.864092 1015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.864259 1000 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:10:46.864341 1000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "0774f62eea22d757dc11681c459ecb83"
server_key_iv: "5114b7752327ee65b83e04cbd62785b5"
server_key_version: "encryptionkey@0"
I20260927 11:10:46.864389 1000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:10:46.876318 1000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:10:46.878435 1000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:10:46.878533 1000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:10:46.878685 1000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:10:46.878964 1000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:10:46.878998 1000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.879058 1000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:10:46.879110 1000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:10:46.884884 1000 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35923
I20260927 11:10:46.884935 1128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35923 every 8 connection(s)
I20260927 11:10:46.885223 1000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
Sep 27 11:10:46 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:10:46.893778 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1000
I20260927 11:10:46.893898 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:10:46.894100 17791 external_mini_cluster.cc:1428] Setting key 2d5edc04c008fd7df63b42366fb4e1a9
I20260927 11:10:46.895290 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.885894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:33369 (local address 127.17.95.254:44343)
0927 11:10:46.885937 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.885939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.886219 (+ 280us) server_negotiation.cc:407] Connection header received
0927 11:10:46.886831 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.886833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.886846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.886909 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:46.888164 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.888723 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.889970 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.890201 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.890285 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.892448 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:46.892450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:46.892450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:46.892457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:46.893785 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.894259 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.894260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.894261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.894327 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.894625 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.894626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.894626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.894765 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:46.894828 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.895156 (+ 328us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.895230 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:10:46.895437 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.885868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:33369)
0927 11:10:46.886082 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.886141 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.886730 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.886972 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.886975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.887783 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:10:46.888047 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.888054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.888836 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.888837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.889876 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.889879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.890297 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.890298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.890426 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.890975 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:10:46.890984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:10:46.892346 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:10:46.893888 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:46.893896 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:46.893899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:46.894179 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:46.894403 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:10:46.894404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:10:46.894405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:10:46.894560 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:10:46.894829 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:10:46.894835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:10:46.894982 (+ 147us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.895117 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.895257 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260927 11:10:46.895838 1129 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44343
I20260927 11:10:46.895942 1129 heartbeater.cc:464] Registering TS with master...
I20260927 11:10:46.896114 1129 heartbeater.cc:511] Master 127.17.95.254:44343 requested a full tablet report, sending...
I20260927 11:10:46.896451 670 ts_manager.cc:194] Registered new tserver with Master: 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923)
I20260927 11:10:46.896807 670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:33369
I20260927 11:10:46.905874 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:10:46.912711 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.907097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.17.95.254:44343)
0927 11:10:46.907165 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.907167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.907345 (+ 178us) server_negotiation.cc:407] Connection header received
0927 11:10:46.907398 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.907398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.907406 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.907450 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:10:46.907974 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.908413 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.909096 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.909294 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.909352 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.910366 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:10:46.910367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:10:46.910368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:10:46.910373 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:10:46.911663 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.911960 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.911961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.911961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.912030 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:10:46.912230 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:10:46.912231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:10:46.912231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:10:46.912398 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:10:46.912469 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.912571 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.912624 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 11:10:46.914242 670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:10:46.915971 670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:10:46.922124 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.918096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.17.95.195:35923)
0927 11:10:46.918230 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.918235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.918246 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:10:46.918283 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.918285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.918300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.918377 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:46.918589 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.919701 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921149 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.921750 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921823 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.921875 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.922038 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.922072 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260927 11:10:46.922152 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.917938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35923 (local address 127.0.0.1:53830)
0927 11:10:46.918074 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.918134 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.918194 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.918455 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.918458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.918460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:46.918519 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.918521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.919806 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.919807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.921074 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.921076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.921834 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.921911 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.921923 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.922011 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.922070 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":52,"threads_started":1}
I20260927 11:10:46.922243 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.918190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.17.95.194:36009)
0927 11:10:46.918370 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.918374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.918501 (+ 127us) server_negotiation.cc:407] Connection header received
0927 11:10:46.918542 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.918546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.918559 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.918971 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:46.919190 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.919947 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921212 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.921767 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921848 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.921940 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.922114 (+ 174us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.922157 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260927 11:10:46.922478 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.917952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:36009 (local address 127.0.0.1:34966)
0927 11:10:46.918429 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.918439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.918490 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.919048 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.919048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.919050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:46.919122 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.919124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.920057 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.920057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.921157 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.921159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.921969 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.922028 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.922030 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.922335 (+ 305us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.922404 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":20,"thread_start_us":72,"threads_started":1}
I20260927 11:10:46.922750 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.917938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41637 (local address 127.0.0.1:43822)
0927 11:10:46.918280 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.918288 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.918320 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.918719 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.918720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.918722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:46.918800 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.918801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.920032 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.920033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.920978 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.920981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.922336 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.922336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.922392 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.922395 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.922442 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.922469 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":35,"thread_start_us":48,"threads_started":1}
I20260927 11:10:46.922796 1063 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6aeb451c744b52917bd2548c203b60 (DEFAULT_TABLE table=test-table [id=df2e2ce2180249a8a70b7d0cda75d6c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:46.923198 1063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6aeb451c744b52917bd2548c203b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:46.923173 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.918269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.17.95.193:41637)
0927 11:10:46.918448 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.918450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.918457 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:10:46.918507 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.918510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.918524 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.918594 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:46.918879 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.919939 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921056 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.921750 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.921823 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.921875 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.922519 (+ 644us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.922571 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260927 11:10:46.923350 929 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6aeb451c744b52917bd2548c203b60 (DEFAULT_TABLE table=test-table [id=df2e2ce2180249a8a70b7d0cda75d6c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:46.923673 929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6aeb451c744b52917bd2548c203b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:46.923822 795 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6aeb451c744b52917bd2548c203b60 (DEFAULT_TABLE table=test-table [id=df2e2ce2180249a8a70b7d0cda75d6c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:10:46.924217 795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6aeb451c744b52917bd2548c203b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:10:46.926121 1143 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Bootstrap starting.
I20260927 11:10:46.926479 1144 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Bootstrap starting.
I20260927 11:10:46.926767 1143 tablet_bootstrap.cc:654] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:46.927103 1144 tablet_bootstrap.cc:654] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:46.927598 1145 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap starting.
I20260927 11:10:46.927591 1143 log.cc:824] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:46.927670 1144 log.cc:824] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:46.928215 1145 tablet_bootstrap.cc:654] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Neither blocks nor log segments found. Creating new log.
I20260927 11:10:46.928356 1144 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: No bootstrap required, opened a new log
I20260927 11:10:46.928350 1143 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: No bootstrap required, opened a new log
I20260927 11:10:46.928421 1144 ts_tablet_manager.cc:1397] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:10:46.928421 1143 ts_tablet_manager.cc:1397] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:10:46.928715 1145 log.cc:824] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Log is configured to *not* fsync() on all Append() calls
I20260927 11:10:46.929369 1145 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: No bootstrap required, opened a new log
I20260927 11:10:46.929443 1145 ts_tablet_manager.cc:1397] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:10:46.930398 1143 raft_consensus.cc:348] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.930398 1144 raft_consensus.cc:348] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.930577 1144 raft_consensus.cc:374] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:46.930577 1143 raft_consensus.cc:374] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:46.930614 1144 raft_consensus.cc:729] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884a11cb30584a1491ac39dfcb1b7ddc, State: Initialized, Role: FOLLOWER
I20260927 11:10:46.930624 1143 raft_consensus.cc:729] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de7a5e791c74f01b8cfa908b9f2cc8f, State: Initialized, Role: FOLLOWER
I20260927 11:10:46.930733 1144 consensus_queue.cc:260] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.930747 1143 consensus_queue.cc:260] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.930984 1144 ts_tablet_manager.cc:1428] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:10:46.930984 1143 ts_tablet_manager.cc:1428] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:10:46.931161 1129 heartbeater.cc:503] Master 127.17.95.254:44343 was elected leader, sending a full tablet report...
I20260927 11:10:46.931141 1145 raft_consensus.cc:348] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.931283 995 heartbeater.cc:503] Master 127.17.95.254:44343 was elected leader, sending a full tablet report...
I20260927 11:10:46.931308 1145 raft_consensus.cc:374] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:10:46.931347 1145 raft_consensus.cc:729] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae020b2cd044833a31c53b3ec5ad84a, State: Initialized, Role: FOLLOWER
I20260927 11:10:46.931572 1145 consensus_queue.cc:260] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.931890 861 heartbeater.cc:503] Master 127.17.95.254:44343 was elected leader, sending a full tablet report...
I20260927 11:10:46.932032 1145 ts_tablet_manager.cc:1428] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 11:10:46.975123 1150 raft_consensus.cc:482] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:10:46.975232 1150 raft_consensus.cc:504] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.975688 1150 leader_election.cc:288] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637)
I20260927 11:10:46.979463 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.975899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:34231 (local address 127.17.95.193:41637)
0927 11:10:46.975941 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.975943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.976115 (+ 172us) server_negotiation.cc:407] Connection header received
0927 11:10:46.976142 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.976143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.976160 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.976200 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:46.976345 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.976999 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.978496 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.979114 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.979173 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.979181 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.979354 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.979393 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:10:46.979463 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.975893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:38445 (local address 127.17.95.195:35923)
0927 11:10:46.975950 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.975951 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.975960 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:10:46.976012 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.976013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.976020 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.976066 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:10:46.976265 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.977000 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.978496 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.979056 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.979160 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.979171 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.979355 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.979393 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:10:46.979475 997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.975827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35923 (local address 127.17.95.194:38445)
0927 11:10:46.975892 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.975905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.975953 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.976095 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.976096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.976098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:46.976176 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.976177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.977090 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.977090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.978403 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.978407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.979169 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.979170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.979253 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.979255 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.979332 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.979386 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":85}
I20260927 11:10:46.979488 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.975828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41637 (local address 127.17.95.194:34231)
0927 11:10:46.976045 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:46.976053 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:10:46.976102 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:10:46.976219 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:10:46.976219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:10:46.976221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:10:46.976277 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.976278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.977089 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.977090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.978403 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:10:46.978407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.979180 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.979181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:10:46.979248 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.979253 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:10:46.979332 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:10:46.979386 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":107,"thread_start_us":105,"threads_started":1}
I20260927 11:10:46.979838 1083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6aeb451c744b52917bd2548c203b60" candidate_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" is_pre_election: true
I20260927 11:10:46.979951 1083 raft_consensus.cc:2454] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2de7a5e791c74f01b8cfa908b9f2cc8f in term 0.
I20260927 11:10:46.980145 884 leader_election.cc:302] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de7a5e791c74f01b8cfa908b9f2cc8f, 884a11cb30584a1491ac39dfcb1b7ddc; no voters:
I20260927 11:10:46.980289 1150 raft_consensus.cc:2781] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:10:46.980342 1150 raft_consensus.cc:482] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:10:46.980357 1150 raft_consensus.cc:3037] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:46.980288 815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6aeb451c744b52917bd2548c203b60" candidate_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" is_pre_election: true
I20260927 11:10:46.980445 815 raft_consensus.cc:2454] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2de7a5e791c74f01b8cfa908b9f2cc8f in term 0.
I20260927 11:10:46.981113 1150 raft_consensus.cc:504] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.981248 1150 leader_election.cc:288] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [CANDIDATE]: Term 1 election: Requested vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637)
I20260927 11:10:46.981405 1083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6aeb451c744b52917bd2548c203b60" candidate_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
I20260927 11:10:46.981463 1083 raft_consensus.cc:3037] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:46.982146 1083 raft_consensus.cc:2454] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2de7a5e791c74f01b8cfa908b9f2cc8f in term 1.
I20260927 11:10:46.982235 815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6aeb451c744b52917bd2548c203b60" candidate_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae020b2cd044833a31c53b3ec5ad84a"
I20260927 11:10:46.982288 815 raft_consensus.cc:3037] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:10:46.982391 884 leader_election.cc:302] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de7a5e791c74f01b8cfa908b9f2cc8f, 884a11cb30584a1491ac39dfcb1b7ddc; no voters:
I20260927 11:10:46.982496 1150 raft_consensus.cc:2781] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:10:46.982627 1150 raft_consensus.cc:686] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 LEADER]: Becoming Leader. State: Replica: 2de7a5e791c74f01b8cfa908b9f2cc8f, State: Running, Role: LEADER
I20260927 11:10:46.982743 1150 consensus_queue.cc:237] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:10:46.983227 815 raft_consensus.cc:2454] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2de7a5e791c74f01b8cfa908b9f2cc8f in term 1.
I20260927 11:10:46.983577 669 catalog_manager.cc:5495] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194). New cstate: current_term: 1 leader_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } health_report { overall_health: HEALTHY } } }
W20260927 11:10:46.989735 996 tablet.cc:2369] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:10:46.995549 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:46.993372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.17.95.194:36009)
0927 11:10:46.993423 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 11:10:46.993425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:10:46.993433 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:10:46.993466 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:10:46.993466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:10:46.993481 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:10:46.993538 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:10:46.993676 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.994097 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.994807 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:10:46.994979 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:10:46.995028 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:10:46.995257 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:10:46.995342 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:10:46.995401 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 11:10:46.995432 (+ 31us) server_negotiation.cc:299] Negotiation successful
0927 11:10:46.995448 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:10:46.997143 1083 raft_consensus.cc:1261] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Refusing update from remote peer 2de7a5e791c74f01b8cfa908b9f2cc8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:46.997167 815 raft_consensus.cc:1261] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Refusing update from remote peer 2de7a5e791c74f01b8cfa908b9f2cc8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:10:46.997505 1153 consensus_queue.cc:1035] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:46.997615 1150 consensus_queue.cc:1035] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:10:46.999928 1157 mvcc.cc:205] Tried to move back new op lower bound from 7333918502897946624 to 7333918502841929728. Current Snapshot: MvccSnapshot[applied={T|T < 7333918502897946624}]
I20260927 11:10:47.000349 1158 mvcc.cc:205] Tried to move back new op lower bound from 7333918502897946624 to 7333918502841929728. Current Snapshot: MvccSnapshot[applied={T|T < 7333918502897946624}]
I20260927 11:10:47.001885 670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:47.001971 670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:10:47.002628 670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Sending DeleteTablet for 3 replicas of tablet 0a6aeb451c744b52917bd2548c203b60
I20260927 11:10:47.002954 929 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6aeb451c744b52917bd2548c203b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:46 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:34966
I20260927 11:10:47.002985 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 732
I20260927 11:10:47.003508 1063 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6aeb451c744b52917bd2548c203b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:46 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:53830
I20260927 11:10:47.003541 1166 tablet_replica.cc:330] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: stopping tablet replica
I20260927 11:10:47.003703 1166 raft_consensus.cc:2229] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:10:47.003721 1167 tablet_replica.cc:330] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: stopping tablet replica
I20260927 11:10:47.003811 1167 raft_consensus.cc:2229] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:10:47.003890 1166 raft_consensus.cc:2258] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:47.003911 1167 raft_consensus.cc:2258] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:10:47.004293 1166 ts_tablet_manager.cc:1905] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:47.004621 1167 ts_tablet_manager.cc:1905] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:10:47.006234 1166 ts_tablet_manager.cc:1918] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:47.006353 1166 log.cc:1196] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/0a6aeb451c744b52917bd2548c203b60
I20260927 11:10:47.006471 1167 ts_tablet_manager.cc:1918] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:10:47.006544 1167 log.cc:1196] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/0a6aeb451c744b52917bd2548c203b60
I20260927 11:10:47.006811 1167 ts_tablet_manager.cc:1939] T 0a6aeb451c744b52917bd2548c203b60 P 884a11cb30584a1491ac39dfcb1b7ddc: Deleting consensus metadata
I20260927 11:10:47.006884 1166 ts_tablet_manager.cc:1939] T 0a6aeb451c744b52917bd2548c203b60 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Deleting consensus metadata
I20260927 11:10:47.007197 656 catalog_manager.cc:4841] TS 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923): tablet 0a6aeb451c744b52917bd2548c203b60 (table test-table [id=df2e2ce2180249a8a70b7d0cda75d6c5]) successfully deleted
I20260927 11:10:47.007501 655 catalog_manager.cc:4841] TS 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009): tablet 0a6aeb451c744b52917bd2548c203b60 (table test-table [id=df2e2ce2180249a8a70b7d0cda75d6c5]) successfully deleted
W20260927 11:10:47.008476 657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:41637 (error 108)
I20260927 11:10:47.008940 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:10:47.008808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41637 (local address 127.0.0.1:43832)
0927 11:10:47.008866 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 11:10:47.008907 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 11:10:47.009034 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 866
W20260927 11:10:47.009177 657 catalog_manager.cc:4584] TS 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a6aeb451c744b52917bd2548c203b60: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41637: connect: Connection refused (error 111)
I20260927 11:10:47.013692 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1000
I20260927 11:10:47.018059 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 638
I20260927 11:11:07.023303 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44343
--webserver_interface=127.17.95.254
--webserver_port=39261
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:44343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:07.101080 1169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:07.101281 1169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:07.101301 1169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:07.102417 1169 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 11:11:07.102461 1169 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 11:11:07.102473 1169 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 11:11:07.102485 1169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:07.102499 1169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:07.102511 1169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:07.102562 1169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:07.102587 1169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:07.104120 1169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:44343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:07.104368 1169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:07.104573 1169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:11:07.113125 1169 server_base.cc:1034] running on GCE node
W20260927 11:11:07.113119 1178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:07.113119 1176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:07.113168 1175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.113617 1169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:07.113912 1169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:07.115044 1169 hybrid_clock.cc:648] HybridClock initialized: now 1790507467115026 us; error 34 us; skew 500 ppm
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:07.117610 1169 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:11:07.118026 1169 webserver.cc:460] Webserver started at http://127.17.95.254:39261/ using document root <none> and password file <none>
I20260927 11:11:07.118255 1169 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:07.118319 1169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:07.119724 1169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:07.120363 1185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.120520 1169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:07.120625 1169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "1e7eaebc174c4d2cb90baa51381f01f9"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "649164aa2400c4ff4e1a6206c9eff6b7"
server_key_iv: "002d5fbb0e2bc920d3459b7a00908370"
server_key_version: "encryptionkey@0"
I20260927 11:11:07.120982 1169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:07.135588 1169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:07.137851 1169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:07.137989 1169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:07.141768 1169 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:44343
I20260927 11:11:07.141825 1237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:44343 every 8 connection(s)
I20260927 11:11:07.142227 1169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:07.143823 1238 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:11:07.144469 1238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap starting.
I20260927 11:11:07.148061 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1169
I20260927 11:11:07.148415 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:41637
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:44343
--webserver_port=33659
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:11:07.153352 1238 log.cc:824] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:07.155136 1238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:11:07.155427 1238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap complete.
I20260927 11:11:07.157552 1238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:07.157792 1238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7eaebc174c4d2cb90baa51381f01f9, State: Initialized, Role: FOLLOWER
I20260927 11:11:07.157985 1238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:07.158077 1238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:07.158130 1238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:07.158176 1238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 1 FOLLOWER]: Advancing to term 2
I20260927 11:11:07.159135 1238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:07.159251 1238 leader_election.cc:302] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e7eaebc174c4d2cb90baa51381f01f9; no voters:
I20260927 11:11:07.159415 1238 leader_election.cc:288] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 11:11:07.159458 1242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 11:11:07.159612 1242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 LEADER]: Becoming Leader. State: Replica: 1e7eaebc174c4d2cb90baa51381f01f9, State: Running, Role: LEADER
I20260927 11:11:07.159674 1238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:07.159787 1242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:07.160116 1242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e7eaebc174c4d2cb90baa51381f01f9. Latest consensus state: current_term: 2 leader_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } } }
I20260927 11:11:07.160132 1244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } } }
I20260927 11:11:07.160306 1244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:07.160645 1242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:07.160764 1251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:07.161460 1251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:07.161584 1251 catalog_manager.cc:1259] Loaded cluster ID: 6546bbf76bc34293a358473b5c24373d
I20260927 11:11:07.161630 1251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:07.162470 1251 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:07.162645 1251 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Loaded TSK: 0
I20260927 11:11:07.169951 1251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Generated new TSK 1
I20260927 11:11:07.170044 1251 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 11:11:07.247188 1240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:07.247393 1240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:07.247418 1240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:07.248823 1240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:07.249038 1240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:07.249075 1240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:07.250720 1240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:41637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=33659
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:07.250947 1240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:07.251237 1240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:07.256549 1266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:07.256557 1265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.256790 1240 server_base.cc:1034] running on GCE node
W20260927 11:11:07.256546 1268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.257063 1240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:07.257339 1240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:07.258524 1240 hybrid_clock.cc:648] HybridClock initialized: now 1790507467258495 us; error 43 us; skew 500 ppm
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:07.261672 1240 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:11:07.262200 1240 webserver.cc:460] Webserver started at http://127.17.95.193:33659/ using document root <none> and password file <none>
I20260927 11:11:07.262463 1240 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:07.262545 1240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:07.264348 1240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:07.265157 1275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.265364 1240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:07.265467 1240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7ae020b2cd044833a31c53b3ec5ad84a"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "964aefa2b2183b32568a9dfa35c3376d"
server_key_iv: "61ce5c898fe856025bea5dc9041369c7"
server_key_version: "encryptionkey@0"
I20260927 11:11:07.265821 1240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:07.273797 1240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:07.276103 1240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:07.276255 1240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:07.276476 1240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:07.276961 1282 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 11:11:07.278811 1240 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 11:11:07.278877 1240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 11:11:07.278921 1240 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 11:11:07.279587 1240 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 11:11:07.279642 1240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 11:11:07.279685 1282 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap starting.
I20260927 11:11:07.287043 1240 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41637
I20260927 11:11:07.287446 1240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:07.290184 1282 log.cc:824] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:07.290484 1389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41637 every 8 connection(s)
I20260927 11:11:07.292726 1282 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:11:07.293020 1282 tablet_bootstrap.cc:492] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap complete.
I20260927 11:11:07.293767 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1240
I20260927 11:11:07.294044 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:36009
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:44343
--webserver_port=40691
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:11:07.294183 1282 ts_tablet_manager.cc:1397] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260927 11:11:07.295423 1282 raft_consensus.cc:348] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:11:07.295730 1282 raft_consensus.cc:729] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae020b2cd044833a31c53b3ec5ad84a, State: Initialized, Role: FOLLOWER
I20260927 11:11:07.295884 1282 consensus_queue.cc:260] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } }
I20260927 11:11:07.296166 1282 ts_tablet_manager.cc:1428] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:07.301069 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.288134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:56957)
0927 11:11:07.290494 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.290534 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.291006 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.291594 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.291597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.291888 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:07.292083 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.292088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.293295 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.293297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.294498 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.294501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.295509 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.295510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.295664 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.296313 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:07.296322 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:07.297585 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:07.299505 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:07.299511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:07.299513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:07.299734 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:07.299992 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:07.299992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:07.299993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:07.300094 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:07.300463 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:07.300470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:07.300603 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.300707 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.300877 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2320,"thread_start_us":94,"threads_started":1}
I20260927 11:11:07.301106 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.288213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:56957 (local address 127.17.95.254:44343)
0927 11:11:07.291223 (+ 3010us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.291232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.291268 (+ 36us) server_negotiation.cc:407] Connection header received
0927 11:11:07.291379 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.291383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.291422 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.291590 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:07.292197 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.293201 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.294598 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.295322 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.295414 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.297686 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:07.297690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:07.297691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:07.297701 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:07.299396 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.299828 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.299832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.299832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.299919 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.300162 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.300163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.300163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.300388 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:07.300469 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.300771 (+ 302us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.300894 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2958,"thread_start_us":87,"threads_started":1}
I20260927 11:11:07.301993 1390 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44343
I20260927 11:11:07.302091 1390 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:07.302254 1390 heartbeater.cc:511] Master 127.17.95.254:44343 requested a full tablet report, sending...
I20260927 11:11:07.302910 1202 ts_manager.cc:194] Registered new tserver with Master: 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637)
I20260927 11:11:07.304091 1202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:56957
I20260927 11:11:07.304926 1390 heartbeater.cc:503] Master 127.17.95.254:44343 was elected leader, sending a full tablet report...
I20260927 11:11:07.310149 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.303712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.17.95.193:41637)
0927 11:11:07.307181 (+ 3469us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.307195 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.307205 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:11:07.307239 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.307241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.307253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.307384 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:07.307593 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.308191 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.309319 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.309738 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.309830 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.309882 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.310047 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.310092 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3438,"thread_start_us":65,"threads_started":1}
I20260927 11:11:07.310160 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.303486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41637 (local address 127.0.0.1:49612)
0927 11:11:07.304426 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.304459 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.304518 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.307393 (+ 2875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.307396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.307398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:07.307490 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.307491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.308273 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.308274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.309229 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.309232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.309830 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.309831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.309928 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.309940 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.310030 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.310081 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":89,"threads_started":1}
I20260927 11:11:07.310599 1324 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6aeb451c744b52917bd2548c203b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:10:46 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49612
I20260927 11:11:07.311031 1399 tablet_replica.cc:330] stopping tablet replica
I20260927 11:11:07.311161 1399 raft_consensus.cc:2229] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:07.311228 1399 raft_consensus.cc:2258] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:07.311501 1399 ts_tablet_manager.cc:1905] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:07.313395 1399 ts_tablet_manager.cc:1918] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:07.313480 1399 log.cc:1196] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal/wals/0a6aeb451c744b52917bd2548c203b60
I20260927 11:11:07.313727 1399 ts_tablet_manager.cc:1939] T 0a6aeb451c744b52917bd2548c203b60 P 7ae020b2cd044833a31c53b3ec5ad84a: Deleting consensus metadata
I20260927 11:11:07.314199 1189 catalog_manager.cc:4841] TS 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637): tablet 0a6aeb451c744b52917bd2548c203b60 (table test-table [id=df2e2ce2180249a8a70b7d0cda75d6c5]) successfully deleted
W20260927 11:11:07.384825 1395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:07.385023 1395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:07.385047 1395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:07.386133 1395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:07.386171 1395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:07.386197 1395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:07.387550 1395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:36009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=40691
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:07.387746 1395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:07.387916 1395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:07.392863 1405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.392988 1395 server_base.cc:1034] running on GCE node
W20260927 11:11:07.392879 1407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:07.392871 1404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.393337 1395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:07.393532 1395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:07.394665 1395 hybrid_clock.cc:648] HybridClock initialized: now 1790507467394648 us; error 34 us; skew 500 ppm
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:07.397020 1395 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:11:07.397384 1395 webserver.cc:460] Webserver started at http://127.17.95.194:40691/ using document root <none> and password file <none>
I20260927 11:11:07.397550 1395 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:07.397610 1395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:07.398886 1395 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:07.399470 1414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.399639 1395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.399749 1395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "7d8c122f21662276d56db0ad9e1ac7cc"
server_key_iv: "4603732d6a209b1301e6cf5916b93f69"
server_key_version: "encryptionkey@0"
I20260927 11:11:07.400087 1395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:07.424189 1395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:07.426476 1395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:07.426645 1395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:07.426873 1395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:07.427182 1395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:07.427234 1395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.427268 1395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:07.427291 1395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.433291 1395 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:36009
I20260927 11:11:07.433326 1527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:36009 every 8 connection(s)
I20260927 11:11:07.433670 1395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:07.440793 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1395
I20260927 11:11:07.441102 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:35923
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:44343
--webserver_port=35441
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:11:07.443176 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.434375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:40119 (local address 127.17.95.254:44343)
0927 11:11:07.434429 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.434431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.434611 (+ 180us) server_negotiation.cc:407] Connection header received
0927 11:11:07.435160 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.435162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.435179 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.435250 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:07.436131 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.436647 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.437654 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.437847 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.437981 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.440149 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:07.440150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:07.440150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:07.440159 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:07.441788 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.442196 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.442197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.442198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.442261 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.442511 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.442512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.442513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.442629 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:07.442699 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.442961 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.443048 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:11:07.443318 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.434343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:40119)
0927 11:11:07.434490 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.434529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.435034 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.435317 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.435320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.435838 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:07.436030 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.436035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.436748 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.436749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.437564 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.437567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.438033 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.438035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.438175 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.438601 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:07.438609 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:07.440013 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:07.441886 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:07.441892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:07.441895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:07.442105 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:07.442339 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:07.442340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:07.442340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:07.442438 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:07.442702 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:07.442708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:07.442833 (+ 125us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.442931 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.443104 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260927 11:11:07.443763 1528 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44343
I20260927 11:11:07.443868 1528 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:07.444068 1528 heartbeater.cc:511] Master 127.17.95.254:44343 requested a full tablet report, sending...
I20260927 11:11:07.444427 1202 ts_manager.cc:194] Registered new tserver with Master: 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
I20260927 11:11:07.444794 1202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:40119
W20260927 11:11:07.521468 1531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:07.521675 1531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:07.521700 1531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:07.522769 1531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:07.522809 1531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:07.522843 1531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:07.524389 1531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:35923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=35441
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:07.524581 1531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:07.524742 1531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:07.529680 1536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.529764 1531 server_base.cc:1034] running on GCE node
W20260927 11:11:07.529690 1537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:07.529747 1539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:07.530061 1531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:07.530334 1531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:07.531488 1531 hybrid_clock.cc:648] HybridClock initialized: now 1790507467531483 us; error 33 us; skew 500 ppm
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:07.533777 1531 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:11:07.534212 1531 webserver.cc:460] Webserver started at http://127.17.95.195:35441/ using document root <none> and password file <none>
I20260927 11:11:07.534425 1531 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:07.534505 1531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:07.535892 1531 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:11:07.536470 1546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.536631 1531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:11:07.536721 1531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "0774f62eea22d757dc11681c459ecb83"
server_key_iv: "5114b7752327ee65b83e04cbd62785b5"
server_key_version: "encryptionkey@0"
I20260927 11:11:07.537025 1531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:07.548223 1531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:07.550000 1531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:07.550158 1531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:07.550369 1531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:07.550652 1531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:07.550705 1531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.550745 1531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:07.550778 1531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.556522 1531 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35923
I20260927 11:11:07.556561 1659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35923 every 8 connection(s)
I20260927 11:11:07.556870 1531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
Sep 27 11:11:07 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507467, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:07.566059 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.557593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:39735 (local address 127.17.95.254:44343)
0927 11:11:07.557668 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.557670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.557816 (+ 146us) server_negotiation.cc:407] Connection header received
0927 11:11:07.558252 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.558253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.558263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.558327 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:07.559227 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.559850 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.560965 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.561174 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.561244 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.563412 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:07.563413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:07.563414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:07.563420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:07.564870 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.565229 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.565230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.565230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.565279 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.565485 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.565485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.565485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.565616 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:07.565665 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.565937 (+ 272us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.565997 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:11:07.566322 1662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.557556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:39735)
0927 11:11:07.557694 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.557731 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.558178 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.558379 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.558382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.558904 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:07.559148 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.559158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.559938 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.559939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.560849 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.560852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.561256 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.561257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.561381 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.561828 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:07.561835 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:07.563322 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:07.564941 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:07.564945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:07.564947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:07.565144 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:07.565337 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:07.565338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:07.565338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:07.565432 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:07.565696 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:07.565702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:07.565820 (+ 118us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.565917 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.566036 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260927 11:11:07.566425 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1531
I20260927 11:11:07.566872 1660 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44343
I20260927 11:11:07.566977 1660 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:07.567189 1660 heartbeater.cc:511] Master 127.17.95.254:44343 requested a full tablet report, sending...
I20260927 11:11:07.567688 1202 ts_manager.cc:194] Registered new tserver with Master: 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923)
I20260927 11:11:07.568194 1202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:39735
I20260927 11:11:07.573171 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.566833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.17.95.254:44343)
0927 11:11:07.566872 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.566874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.567100 (+ 226us) server_negotiation.cc:407] Connection header received
0927 11:11:07.567199 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.567200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.567212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.567271 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:07.568119 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.568731 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.569548 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.569723 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.569776 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.570860 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:07.570861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:07.570861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:07.570867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:07.572304 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.572609 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.572610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.572610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.572660 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.572801 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.572801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.572801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.572883 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:07.572999 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.573025 (+ 26us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.573083 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:11:07.573959 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:07.576459 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.574321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.17.95.254:44343)
0927 11:11:07.574363 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.574365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.574443 (+ 78us) server_negotiation.cc:407] Connection header received
0927 11:11:07.574500 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.574501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.574528 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.574609 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:07.574703 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.575180 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.575876 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.576041 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.576109 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.576250 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:07.576285 (+ 35us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 11:11:07.576398 (+ 113us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52978: token expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 11:11:07.576514 1393 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52978: token expired
W20260927 11:11:07.576610 1134 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.95.254:44343)
I20260927 11:11:07.582306 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.576896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.17.95.254:44343)
0927 11:11:07.576937 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.576938 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.576946 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:11:07.576984 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.576984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.576991 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.577043 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:07.577466 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.577902 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.578519 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.578724 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.578775 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.579874 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:07.579875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:07.579875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:07.579881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:07.581330 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.581612 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.581612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.581613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.581663 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:07.581858 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:07.581859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:07.581859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:07.581983 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:07.582078 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.582153 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.582210 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:11:07.583453 1202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:07.585023 1202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:07.587221 1324 tablet_service.cc:1467] Processing CreateTablet for tablet a5021c5e8a524a27baaa9d397ea705a6 (DEFAULT_TABLE table=test-table [id=bde3972e7fc9439ea1576fcbda6f6c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:07.587467 1324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5021c5e8a524a27baaa9d397ea705a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:07.589124 1282 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap starting.
I20260927 11:11:07.589457 1282 tablet_bootstrap.cc:654] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:07.590019 1282 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: No bootstrap required, opened a new log
I20260927 11:11:07.590065 1282 ts_tablet_manager.cc:1397] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:07.590163 1282 raft_consensus.cc:348] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.590235 1282 raft_consensus.cc:374] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:07.590270 1282 raft_consensus.cc:729] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae020b2cd044833a31c53b3ec5ad84a, State: Initialized, Role: FOLLOWER
I20260927 11:11:07.590317 1282 consensus_queue.cc:260] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.590397 1282 ts_tablet_manager.cc:1428] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:07.591049 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.587092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.17.95.195:35923)
0927 11:11:07.587294 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.587298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.587322 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:11:07.587379 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.587381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.587398 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.587486 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:07.587613 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.588632 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.589972 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.590685 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.590785 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.590841 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.590928 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.590964 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
I20260927 11:11:07.591064 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.586931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35923 (local address 127.0.0.1:59632)
0927 11:11:07.586977 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.586988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.587021 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.587501 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.587503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.587505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:07.587544 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.587546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.588743 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.588744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.589816 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.589820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.590796 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.590797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.590856 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.590859 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.590914 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.590961 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":35}
I20260927 11:11:07.591266 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.586931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:36009 (local address 127.0.0.1:43442)
0927 11:11:07.587123 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.587131 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.587171 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.587712 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.587712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.587714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:07.587753 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.587754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.588816 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.588817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.589813 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.589817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.590809 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.590810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.590874 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.590878 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.590913 (+ 35us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.590971 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":13,"thread_start_us":67,"threads_started":1}
I20260927 11:11:07.591398 1667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.587273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.17.95.194:36009)
0927 11:11:07.587524 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.587530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.587540 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:11:07.587578 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.587581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.587624 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.587708 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:07.587801 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.588664 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.589931 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.590696 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.591131 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.591198 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.591226 (+ 28us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.591265 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":133,"threads_started":1}
I20260927 11:11:07.592023 1594 tablet_service.cc:1467] Processing CreateTablet for tablet a5021c5e8a524a27baaa9d397ea705a6 (DEFAULT_TABLE table=test-table [id=bde3972e7fc9439ea1576fcbda6f6c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:07.592367 1594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5021c5e8a524a27baaa9d397ea705a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:07.594954 1669 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Bootstrap starting.
I20260927 11:11:07.595280 1462 tablet_service.cc:1467] Processing CreateTablet for tablet a5021c5e8a524a27baaa9d397ea705a6 (DEFAULT_TABLE table=test-table [id=bde3972e7fc9439ea1576fcbda6f6c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:07.595597 1462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5021c5e8a524a27baaa9d397ea705a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:07.595690 1669 tablet_bootstrap.cc:654] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:07.596176 1669 log.cc:824] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:07.596909 1669 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: No bootstrap required, opened a new log
I20260927 11:11:07.596998 1669 ts_tablet_manager.cc:1397] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:07.598258 1671 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Bootstrap starting.
I20260927 11:11:07.598943 1671 tablet_bootstrap.cc:654] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:07.598839 1669 raft_consensus.cc:348] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.599028 1669 raft_consensus.cc:374] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:07.599066 1669 raft_consensus.cc:729] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884a11cb30584a1491ac39dfcb1b7ddc, State: Initialized, Role: FOLLOWER
I20260927 11:11:07.599236 1669 consensus_queue.cc:260] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.599583 1660 heartbeater.cc:503] Master 127.17.95.254:44343 was elected leader, sending a full tablet report...
I20260927 11:11:07.599603 1671 log.cc:824] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:07.599601 1669 ts_tablet_manager.cc:1428] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:07.600394 1671 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: No bootstrap required, opened a new log
I20260927 11:11:07.600474 1671 ts_tablet_manager.cc:1397] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:07.601861 1671 raft_consensus.cc:348] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.601996 1671 raft_consensus.cc:374] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:07.602018 1671 raft_consensus.cc:729] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de7a5e791c74f01b8cfa908b9f2cc8f, State: Initialized, Role: FOLLOWER
I20260927 11:11:07.602126 1671 consensus_queue.cc:260] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.602335 1671 ts_tablet_manager.cc:1428] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:07.602413 1528 heartbeater.cc:503] Master 127.17.95.254:44343 was elected leader, sending a full tablet report...
I20260927 11:11:07.630496 1396 raft_consensus.cc:482] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:07.630616 1396 raft_consensus.cc:504] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.630934 1396 leader_election.cc:288] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
I20260927 11:11:07.634233 1667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.631197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:46427 (local address 127.17.95.194:36009)
0927 11:11:07.631241 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.631243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.631380 (+ 137us) server_negotiation.cc:407] Connection header received
0927 11:11:07.631410 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.631411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.631422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.631491 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:07.631624 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.632335 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.633424 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.633839 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.633893 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.633900 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.634102 (+ 202us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.634146 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 11:11:07.634277 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.631162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:60509 (local address 127.17.95.195:35923)
0927 11:11:07.631215 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.631217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.631225 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:11:07.631250 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.631251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.631261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.631317 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:07.631896 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.632586 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.633516 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.633865 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.633961 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.633969 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.634151 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.634210 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:11:07.634270 1675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.631112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:36009 (local address 127.17.95.193:46427)
0927 11:11:07.631304 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.631312 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.631361 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.631499 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.631499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.631501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:07.631554 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.631556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.632400 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.632401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.633325 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.633328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.633929 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.633930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.634024 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.634028 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.634067 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.634097 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":2,"thread_start_us":121,"threads_started":1}
I20260927 11:11:07.634344 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.631054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35923 (local address 127.17.95.193:60509)
0927 11:11:07.631139 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.631152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:07.631196 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:07.631341 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:07.631342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:07.631344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:07.631787 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.631789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.632668 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.632669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.633430 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:07.633432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.633983 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.633984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:07.634050 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.634078 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 11:11:07.634134 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:11:07.634194 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 11:11:07.634721 1614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5021c5e8a524a27baaa9d397ea705a6" candidate_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" is_pre_election: true
I20260927 11:11:07.634809 1482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5021c5e8a524a27baaa9d397ea705a6" candidate_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" is_pre_election: true
I20260927 11:11:07.634887 1614 raft_consensus.cc:2454] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ae020b2cd044833a31c53b3ec5ad84a in term 0.
I20260927 11:11:07.634979 1482 raft_consensus.cc:2454] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ae020b2cd044833a31c53b3ec5ad84a in term 0.
I20260927 11:11:07.635143 1278 leader_election.cc:302] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ae020b2cd044833a31c53b3ec5ad84a, 884a11cb30584a1491ac39dfcb1b7ddc; no voters:
I20260927 11:11:07.635273 1396 raft_consensus.cc:2781] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:07.635339 1396 raft_consensus.cc:482] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:07.635365 1396 raft_consensus.cc:3037] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:07.636102 1396 raft_consensus.cc:504] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.636248 1396 leader_election.cc:288] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 1 election: Requested vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
I20260927 11:11:07.636396 1614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5021c5e8a524a27baaa9d397ea705a6" candidate_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
I20260927 11:11:07.636406 1482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5021c5e8a524a27baaa9d397ea705a6" candidate_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f"
I20260927 11:11:07.636475 1614 raft_consensus.cc:3037] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:07.636492 1482 raft_consensus.cc:3037] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:07.637192 1482 raft_consensus.cc:2454] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ae020b2cd044833a31c53b3ec5ad84a in term 1.
I20260927 11:11:07.637239 1614 raft_consensus.cc:2454] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ae020b2cd044833a31c53b3ec5ad84a in term 1.
I20260927 11:11:07.637395 1278 leader_election.cc:302] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ae020b2cd044833a31c53b3ec5ad84a, 884a11cb30584a1491ac39dfcb1b7ddc; no voters:
I20260927 11:11:07.637490 1396 raft_consensus.cc:2781] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:07.637641 1396 raft_consensus.cc:686] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 LEADER]: Becoming Leader. State: Replica: 7ae020b2cd044833a31c53b3ec5ad84a, State: Running, Role: LEADER
I20260927 11:11:07.637727 1396 consensus_queue.cc:237] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:07.638507 1202 catalog_manager.cc:5495] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193). New cstate: current_term: 1 leader_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } health_report { overall_health: HEALTHY } } }
I20260927 11:11:07.648387 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.645912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.17.95.193:41637)
0927 11:11:07.645966 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:07.645968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:07.645977 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:11:07.646050 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:07.646051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:07.646076 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:07.646165 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:07.646296 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.646752 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.647599 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:07.647786 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:07.647877 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:07.648050 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:07.648150 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:07.648241 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:07.648287 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 11:11:07.648308 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 11:11:07.649751 1482 raft_consensus.cc:1261] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Refusing update from remote peer 7ae020b2cd044833a31c53b3ec5ad84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:07.649773 1614 raft_consensus.cc:1261] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Refusing update from remote peer 7ae020b2cd044833a31c53b3ec5ad84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:07.650082 1396 consensus_queue.cc:1035] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:07.650224 1676 consensus_queue.cc:1035] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:07.652199 1679 mvcc.cc:205] Tried to move back new op lower bound from 7333918587491127296 to 7333918587444649984. Current Snapshot: MvccSnapshot[applied={T|T < 7333918587491127296}]
I20260927 11:11:07.652519 1681 mvcc.cc:205] Tried to move back new op lower bound from 7333918587491127296 to 7333918587444649984. Current Snapshot: MvccSnapshot[applied={T|T < 7333918587491127296}]
I20260927 11:11:07.653129 1680 mvcc.cc:205] Tried to move back new op lower bound from 7333918587491127296 to 7333918587444649984. Current Snapshot: MvccSnapshot[applied={T|T < 7333918587491127296}]
I20260927 11:11:07.654939 1202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:07.655027 1202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:07.655763 1202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Sending DeleteTablet for 3 replicas of tablet a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:07.656075 1324 tablet_service.cc:1514] Processing DeleteTablet for tablet a5021c5e8a524a27baaa9d397ea705a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49612
I20260927 11:11:07.656068 1462 tablet_service.cc:1514] Processing DeleteTablet for tablet a5021c5e8a524a27baaa9d397ea705a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:43442
I20260927 11:11:07.656203 1594 tablet_service.cc:1514] Processing DeleteTablet for tablet a5021c5e8a524a27baaa9d397ea705a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:07 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:59632
I20260927 11:11:07.656245 1399 tablet_replica.cc:330] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: stopping tablet replica
I20260927 11:11:07.656292 1689 tablet_replica.cc:330] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: stopping tablet replica
I20260927 11:11:07.656317 1399 raft_consensus.cc:2229] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:11:07.656368 1689 raft_consensus.cc:2229] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:07.656390 1399 raft_consensus.cc:2258] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:07.656503 1690 tablet_replica.cc:330] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: stopping tablet replica
I20260927 11:11:07.656508 1689 raft_consensus.cc:2258] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:07.656560 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1240
I20260927 11:11:07.656733 1690 raft_consensus.cc:2229] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:07.657105 1689 ts_tablet_manager.cc:1905] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:07.658289 1690 raft_consensus.cc:2258] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:07.658715 1690 ts_tablet_manager.cc:1905] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:07.658778 1689 ts_tablet_manager.cc:1918] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:07.658882 1689 log.cc:1196] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:07.659195 1689 ts_tablet_manager.cc:1939] T a5021c5e8a524a27baaa9d397ea705a6 P 2de7a5e791c74f01b8cfa908b9f2cc8f: Deleting consensus metadata
I20260927 11:11:07.659816 1187 catalog_manager.cc:4841] TS 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009): tablet a5021c5e8a524a27baaa9d397ea705a6 (table test-table [id=bde3972e7fc9439ea1576fcbda6f6c08]) successfully deleted
I20260927 11:11:07.660629 1690 ts_tablet_manager.cc:1918] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:07.660698 1690 log.cc:1196] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:07.660987 1690 ts_tablet_manager.cc:1939] T a5021c5e8a524a27baaa9d397ea705a6 P 884a11cb30584a1491ac39dfcb1b7ddc: Deleting consensus metadata
I20260927 11:11:07.661415 1188 catalog_manager.cc:4841] TS 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923): tablet a5021c5e8a524a27baaa9d397ea705a6 (table test-table [id=bde3972e7fc9439ea1576fcbda6f6c08]) successfully deleted
W20260927 11:11:07.663471 1189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:41637 (error 108)
I20260927 11:11:07.663775 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1395
I20260927 11:11:07.663938 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:07.663802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41637 (local address 127.0.0.1:49628)
0927 11:11:07.663861 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:07.663901 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 11:11:07.664459 1189 catalog_manager.cc:4584] TS 7ae020b2cd044833a31c53b3ec5ad84a (127.17.95.193:41637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5021c5e8a524a27baaa9d397ea705a6: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41637: connect: Connection refused (error 111)
I20260927 11:11:07.669037 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1531
I20260927 11:11:07.673362 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1169
I20260927 11:11:27.678565 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44343
--webserver_interface=127.17.95.254
--webserver_port=39261
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:44343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:27.757397 1692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:27.757607 1692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:27.757644 1692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:27.758687 1692 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 11:11:27.758737 1692 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 11:11:27.758761 1692 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 11:11:27.758777 1692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:27.758808 1692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:27.758834 1692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:27.758857 1692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:27.758880 1692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:27.760258 1692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:44343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:27.760506 1692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:27.760694 1692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:27.765170 1700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:27.765164 1697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:27.765224 1698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:27.765316 1692 server_base.cc:1034] running on GCE node
I20260927 11:11:27.765581 1692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:27.765853 1692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:27.766997 1692 hybrid_clock.cc:648] HybridClock initialized: now 1790507487766987 us; error 26 us; skew 500 ppm
Sep 27 11:11:27 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507487, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:27.769421 1692 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:11:27.769866 1692 webserver.cc:460] Webserver started at http://127.17.95.254:39261/ using document root <none> and password file <none>
I20260927 11:11:27.770064 1692 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:27.770109 1692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:27.771378 1692 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:27.772006 1707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:27.772174 1692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:27.772255 1692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "1e7eaebc174c4d2cb90baa51381f01f9"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "649164aa2400c4ff4e1a6206c9eff6b7"
server_key_iv: "002d5fbb0e2bc920d3459b7a00908370"
server_key_version: "encryptionkey@0"
I20260927 11:11:27.772568 1692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:27.784240 1692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:27.785925 1692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:27.786034 1692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:27.789885 1692 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:44343
I20260927 11:11:27.789921 1759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:44343 every 8 connection(s)
I20260927 11:11:27.790279 1692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:27.791769 1760 sys_catalog.cc:263] Verifying existing consensus state
I20260927 11:11:27.792344 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap starting.
I20260927 11:11:27.793118 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1692
I20260927 11:11:27.793378 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:41637
--local_ip_for_outbound_sockets=127.17.95.193
--tserver_master_addrs=127.17.95.254:44343
--webserver_port=33659
--webserver_interface=127.17.95.193
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:11:27.800936 1760 log.cc:824] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:27.803350 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:11:27.803648 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Bootstrap complete.
I20260927 11:11:27.805792 1760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:27.806097 1760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7eaebc174c4d2cb90baa51381f01f9, State: Initialized, Role: FOLLOWER
I20260927 11:11:27.806272 1760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:27.806348 1760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:27.806401 1760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:27.806453 1760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 2 FOLLOWER]: Advancing to term 3
I20260927 11:11:27.807518 1760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:27.807636 1760 leader_election.cc:302] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e7eaebc174c4d2cb90baa51381f01f9; no voters:
I20260927 11:11:27.807842 1760 leader_election.cc:288] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 11:11:27.807881 1764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 11:11:27.808041 1764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [term 3 LEADER]: Becoming Leader. State: Replica: 1e7eaebc174c4d2cb90baa51381f01f9, State: Running, Role: LEADER
I20260927 11:11:27.808162 1760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:27.808187 1764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } }
I20260927 11:11:27.808531 1765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } } }
I20260927 11:11:27.808738 1765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:27.808532 1764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e7eaebc174c4d2cb90baa51381f01f9. Latest consensus state: current_term: 3 leader_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7eaebc174c4d2cb90baa51381f01f9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44343 } } }
I20260927 11:11:27.808915 1764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:27.808976 1773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:27.809863 1773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:27.809994 1773 catalog_manager.cc:1259] Loaded cluster ID: 6546bbf76bc34293a358473b5c24373d
I20260927 11:11:27.810108 1773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:27.811483 1773 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:27.811674 1773 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Loaded TSK: 0
I20260927 11:11:27.811728 1773 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e7eaebc174c4d2cb90baa51381f01f9: Loaded TSK: 1
F20260927 11:11:27.811944 1773 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f75012019ad google::LogMessage::Fail() at ??:0
@ 0x7f750120488a google::LogMessage::SendToLog() at ??:0
@ 0x7f75012014de google::LogMessage::Flush() at ??:0
@ 0x7f7501202a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f750396e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f750396bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f750915b1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f750915b48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f750915b594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f7509121c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f750912bad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f75030cf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f75030c4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7503f286db start_thread at ??:0
@ 0x7f74ffcff61f clone at ??:0
W20260927 11:11:27.890599 1762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:27.890813 1762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:27.890836 1762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:27.892061 1762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:27.892102 1762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:27.892135 1762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:27.893558 1762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:41637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=33659
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:27.893769 1762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:27.893983 1762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:27.898697 1802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:27.898772 1801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:27.898698 1804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:27.899237 1762 server_base.cc:1034] running on GCE node
I20260927 11:11:27.899432 1762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:27.899673 1762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:27.900818 1762 hybrid_clock.cc:648] HybridClock initialized: now 1790507487900808 us; error 25 us; skew 500 ppm
Sep 27 11:11:27 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507487, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:27.903307 1762 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:11:27.903777 1762 webserver.cc:460] Webserver started at http://127.17.95.193:33659/ using document root <none> and password file <none>
I20260927 11:11:27.903983 1762 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:27.904043 1762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:27.905262 1762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:11:27.905880 1811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:27.906070 1762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:11:27.906165 1762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "7ae020b2cd044833a31c53b3ec5ad84a"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "964aefa2b2183b32568a9dfa35c3376d"
server_key_iv: "61ce5c898fe856025bea5dc9041369c7"
server_key_version: "encryptionkey@0"
I20260927 11:11:27.906563 1762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:27.923930 1762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:27.926410 1762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:27.926591 1762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:27.926842 1762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:27.927357 1818 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 11:11:27.928773 1762 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 11:11:27.928836 1762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 11:11:27.928875 1762 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 11:11:27.929760 1762 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 11:11:27.929807 1762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 11:11:27.929876 1818 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap starting.
I20260927 11:11:27.937563 1762 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41637
I20260927 11:11:27.937649 1925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41637 every 8 connection(s)
I20260927 11:11:27.938002 1762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:27.938513 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1762
I20260927 11:11:27.938827 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:36009
--local_ip_for_outbound_sockets=127.17.95.194
--tserver_master_addrs=127.17.95.254:44343
--webserver_port=40691
--webserver_interface=127.17.95.194
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:11:27.939770 1818 log.cc:824] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:27.944149 1818 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 11:11:27.944489 1818 tablet_bootstrap.cc:492] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Bootstrap complete.
I20260927 11:11:27.945698 1818 ts_tablet_manager.cc:1397] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260927 11:11:27.947539 1818 raft_consensus.cc:348] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:27.947938 1818 raft_consensus.cc:729] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae020b2cd044833a31c53b3ec5ad84a, State: Initialized, Role: FOLLOWER
I20260927 11:11:27.948143 1818 consensus_queue.cc:260] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:27.948458 1818 ts_tablet_manager.cc:1428] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260927 11:11:28.007580 1928 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:11:27.938722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:38181)
0927 11:11:27.938917 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:27.938970 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:27.939695 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:28.007386 (+ 67691us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
W20260927 11:11:28.007840 1926 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:44343 (0 consecutive failures): Network error: Failed to ping master at 127.17.95.254:44343: Client connection negotiation failed: client connection to 127.17.95.254:44343: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:11:28.008189 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.008065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:56753)
0927 11:11:28.008117 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.008163 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 11:11:28.008498 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.008425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:41463)
0927 11:11:28.008454 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.008479 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260927 11:11:28.008689 1926 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:11:28.020864 1929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:28.021034 1929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:28.021056 1929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:28.022117 1929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:28.022146 1929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:28.022171 1929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:28.023553 1929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:36009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=40691
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:28.023722 1929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:28.023881 1929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:28.028468 1939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:28.028532 1937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:28.028617 1929 server_base.cc:1034] running on GCE node
W20260927 11:11:28.028716 1936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:28.028903 1929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:28.029074 1929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:28.030239 1929 hybrid_clock.cc:648] HybridClock initialized: now 1790507488030230 us; error 27 us; skew 500 ppm
Sep 27 11:11:28 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:28.032560 1929 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:11:28.033013 1929 webserver.cc:460] Webserver started at http://127.17.95.194:40691/ using document root <none> and password file <none>
I20260927 11:11:28.033254 1929 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:28.033316 1929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:28.034543 1929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:28.035064 1946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:28.035362 1929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:28.035449 1929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "7d8c122f21662276d56db0ad9e1ac7cc"
server_key_iv: "4603732d6a209b1301e6cf5916b93f69"
server_key_version: "encryptionkey@0"
I20260927 11:11:28.035735 1929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:28.045313 1929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:28.047159 1929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:28.047294 1929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:28.047511 1929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:28.047796 1929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:28.047845 1929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:28.047883 1929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:28.047911 1929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:28.053570 1929 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:36009
I20260927 11:11:28.053622 2059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:36009 every 8 connection(s)
I20260927 11:11:28.053925 1929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:11:28.054376 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 1929
I20260927 11:11:28.054642 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:35923
--local_ip_for_outbound_sockets=127.17.95.195
--tserver_master_addrs=127.17.95.254:44343
--webserver_port=35441
--webserver_interface=127.17.95.195
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:11:28.055024 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.054577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:35741)
0927 11:11:28.054752 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.054838 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
W20260927 11:11:28.055397 2060 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:44343 (0 consecutive failures): Network error: Failed to ping master at 127.17.95.254:44343: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
I20260927 11:11:28.055691 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.055576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:49755)
0927 11:11:28.055628 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.055660 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 11:11:28.056025 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.055894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:59097)
0927 11:11:28.055927 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.055956 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260927 11:11:28.056178 2060 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 11:11:28.132223 2063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:28.132401 2063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:28.132423 2063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:28.133545 2063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:28.133585 2063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:28.133613 2063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:28.135000 2063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:35923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=35441
--tserver_master_addrs=127.17.95.254:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:28.135258 2063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:28.135471 2063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:28.140425 2069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:28.140465 2063 server_base.cc:1034] running on GCE node
W20260927 11:11:28.140442 2071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:28.140587 2068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:28.140796 2063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:28.141038 2063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:28.142200 2063 hybrid_clock.cc:648] HybridClock initialized: now 1790507488142193 us; error 33 us; skew 500 ppm
Sep 27 11:11:28 dist-test-slave-6j0v krb5kdc[625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:28.144637 2063 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:11:28.145026 2063 webserver.cc:460] Webserver started at http://127.17.95.195:35441/ using document root <none> and password file <none>
I20260927 11:11:28.145263 2063 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:28.145327 2063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:28.146536 2063 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:28.147037 2078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:28.147214 2063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:28.147295 2063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "884a11cb30584a1491ac39dfcb1b7ddc"
format_stamp: "Formatted at 2026-09-27 11:10:46 on dist-test-slave-6j0v"
server_key: "0774f62eea22d757dc11681c459ecb83"
server_key_iv: "5114b7752327ee65b83e04cbd62785b5"
server_key_version: "encryptionkey@0"
I20260927 11:11:28.147605 2063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:28.200906 2063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:28.202740 2063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:28.202844 2063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:28.203015 2063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:28.203358 2063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:28.203393 2063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:28.203415 2063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:28.203428 2063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:28.209079 2063 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35923
I20260927 11:11:28.209153 2191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35923 every 8 connection(s)
I20260927 11:11:28.209434 2063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:11:28.210175 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2063
I20260927 11:11:28.210564 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.210069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:58451)
0927 11:11:28.210231 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.210311 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
W20260927 11:11:28.211023 2192 heartbeater.cc:644] Failed to heartbeat to 127.17.95.254:44343 (0 consecutive failures): Network error: Failed to ping master at 127.17.95.254:44343: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
I20260927 11:11:28.211325 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.211231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:48723)
0927 11:11:28.211276 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.211300 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 11:11:28.211647 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:28.211508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:45583)
0927 11:11:28.211563 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:28.211627 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 11:11:28.211745 2192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 11:11:29.009235 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.009019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:55273)
0927 11:11:29.009140 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:29.009175 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
I20260927 11:11:29.056661 2197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.056483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:38627)
0927 11:11:29.056583 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:29.056615 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260927 11:11:29.212241 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.212049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:38149)
0927 11:11:29.212164 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:29.212195 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260927 11:11:29.802533 2199 raft_consensus.cc:482] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:29.802676 2199 raft_consensus.cc:504] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:29.802994 2199 leader_election.cc:288] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
Sep 27 11:11:29 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507487, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.194@KRBTEST.COM
Sep 27 11:11:29 dist-test-slave-6j0v krb5kdc[625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507487, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.195@KRBTEST.COM
I20260927 11:11:29.814648 2202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.803327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:38393 (local address 127.17.95.194:36009)
0927 11:11:29.803503 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:29.803517 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:29.803544 (+ 27us) server_negotiation.cc:407] Connection header received
0927 11:11:29.804127 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:29.804131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:29.804195 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:29.804361 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:29.805374 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.806304 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.807455 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.807871 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.807996 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:29.810173 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:29.810177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:29.810177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:29.810193 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:29.812588 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:29.813204 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:29.813207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:29.813207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:29.813332 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:29.813662 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:29.813663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:29.813663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:29.813969 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:29.814062 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:29.814416 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 11:11:29.814541 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":63,"threads_started":1}
I20260927 11:11:29.814649 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.803336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:39629 (local address 127.17.95.195:35923)
0927 11:11:29.803503 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:29.803517 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:29.803544 (+ 27us) server_negotiation.cc:407] Connection header received
0927 11:11:29.804145 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:29.804148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:29.804195 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:29.804362 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:29.805360 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.806344 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.807482 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.807916 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.808054 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:29.810650 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:29.810668 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:29.810668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:29.810680 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:29.812708 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:29.813204 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:29.813207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:29.813207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:29.813332 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:29.813662 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:29.813662 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:29.813663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:29.813969 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:29.814062 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:29.814416 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 11:11:29.814541 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":59,"threads_started":1}
I20260927 11:11:29.814710 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.803178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:36009 (local address 127.17.95.193:38393)
0927 11:11:29.803405 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:29.803415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:29.803463 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:29.804426 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:29.804429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:29.804926 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:29.805204 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:29.805211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.806402 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.806403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:29.807349 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:29.807351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.807971 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.807972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:29.808080 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:29.808581 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:29.808601 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:29.810041 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:29.812676 (+ 2635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:29.812682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:29.812684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:29.813116 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:29.813404 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:29.813404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:29.813405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:29.813585 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:29.814055 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:29.814064 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:29.814220 (+ 156us) client_negotiation.cc:769] Sending connection context
0927 11:11:29.814348 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 11:11:29.814585 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":174,"thread_start_us":83,"threads_started":1}
I20260927 11:11:29.814709 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:29.803178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35923 (local address 127.17.95.193:39629)
0927 11:11:29.803328 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:29.803339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:29.803423 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:29.804426 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:29.804429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:29.804927 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:29.805204 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:29.805211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.806446 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.806448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:29.807353 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:29.807355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:29.808014 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:29.808015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:29.808090 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:29.808754 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:29.808765 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:29.810542 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:29.812784 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:29.812785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:29.812786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:29.813116 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:29.813404 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:29.813404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:29.813405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:29.813592 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:29.814055 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:29.814064 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:29.814220 (+ 156us) client_negotiation.cc:769] Sending connection context
0927 11:11:29.814348 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 11:11:29.814530 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":11,"thread_start_us":71,"threads_started":1}
W20260927 11:11:29.815524 1814 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
W20260927 11:11:29.816004 1813 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:29.816061 1813 leader_election.cc:302] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae020b2cd044833a31c53b3ec5ad84a; no voters: 2de7a5e791c74f01b8cfa908b9f2cc8f, 884a11cb30584a1491ac39dfcb1b7ddc
I20260927 11:11:29.816170 2199 raft_consensus.cc:2726] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:11:30.009837 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:30.009694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:43003)
0927 11:11:30.009745 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:30.009797 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 11:11:30.057313 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:30.057101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:36715)
0927 11:11:30.057218 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:30.057267 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260927 11:11:30.212800 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:30.212617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:54035)
0927 11:11:30.212718 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:30.212760 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
I20260927 11:11:31.010551 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:31.010292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:43277)
0927 11:11:31.010445 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:31.010483 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260927 11:11:31.058012 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:31.057812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:45713)
0927 11:11:31.057941 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:31.057973 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260927 11:11:31.213393 2208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:31.213212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:40107)
0927 11:11:31.213321 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:31.213354 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260927 11:11:31.394932 2209 raft_consensus.cc:482] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:31.395040 2209 raft_consensus.cc:504] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:31.395272 2209 leader_election.cc:288] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
W20260927 11:11:31.395686 1813 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
W20260927 11:11:31.395794 1814 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:31.395848 1814 leader_election.cc:302] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae020b2cd044833a31c53b3ec5ad84a; no voters: 2de7a5e791c74f01b8cfa908b9f2cc8f, 884a11cb30584a1491ac39dfcb1b7ddc
I20260927 11:11:31.395944 2209 raft_consensus.cc:2726] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:11:32.011317 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:32.011128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:46667)
0927 11:11:32.011237 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:32.011273 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260927 11:11:32.058614 2211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:32.058426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:50303)
0927 11:11:32.058540 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:32.058572 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260927 11:11:32.213986 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:32.213796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:59497)
0927 11:11:32.213916 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:32.213947 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
I20260927 11:11:33.012054 2213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:33.011811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:40487)
0927 11:11:33.011977 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:33.012012 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260927 11:11:33.059253 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:33.059022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:35649)
0927 11:11:33.059181 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:33.059214 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260927 11:11:33.214607 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:33.214396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:48991)
0927 11:11:33.214531 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:33.214564 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260927 11:11:34.012812 2216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:34.012583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:36885)
0927 11:11:34.012715 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:34.012755 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260927 11:11:34.059837 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:34.059642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:53547)
0927 11:11:34.059766 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:34.059798 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260927 11:11:34.215492 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:34.215306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:58445)
0927 11:11:34.215418 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:34.215451 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260927 11:11:34.474906 2219 raft_consensus.cc:482] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:34.474995 2219 raft_consensus.cc:504] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:34.475183 2219 leader_election.cc:288] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
W20260927 11:11:34.475507 1814 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
W20260927 11:11:34.475625 1813 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:34.475669 1813 leader_election.cc:302] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae020b2cd044833a31c53b3ec5ad84a; no voters: 2de7a5e791c74f01b8cfa908b9f2cc8f, 884a11cb30584a1491ac39dfcb1b7ddc
I20260927 11:11:34.475759 2219 raft_consensus.cc:2726] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:11:35.013451 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:35.013256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:43507)
0927 11:11:35.013374 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:35.013408 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260927 11:11:35.060405 2221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:35.060229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:36063)
0927 11:11:35.060334 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:35.060365 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260927 11:11:35.216133 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:35.215930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:35829)
0927 11:11:35.216051 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:35.216090 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260927 11:11:36.014137 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:36.013936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:53981)
0927 11:11:36.014048 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:36.014081 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260927 11:11:36.060995 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:36.060809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:41565)
0927 11:11:36.060925 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:36.060958 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
I20260927 11:11:36.216895 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:36.216706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:40891)
0927 11:11:36.216815 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:36.216846 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260927 11:11:37.014772 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:37.014553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:46201)
0927 11:11:37.014690 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:37.014726 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260927 11:11:37.061718 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:37.061482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:49725)
0927 11:11:37.061639 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:37.061675 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":57,"threads_started":1}
I20260927 11:11:37.217509 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:37.217311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:33407)
0927 11:11:37.217433 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:37.217469 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260927 11:11:38.015432 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:38.015250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:55367)
0927 11:11:38.015356 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:38.015389 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260927 11:11:38.062342 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:38.062152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:60379)
0927 11:11:38.062273 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:38.062305 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260927 11:11:38.218194 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:38.217928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:40907)
0927 11:11:38.218122 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:38.218155 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":52,"threads_started":1}
I20260927 11:11:38.322963 2232 raft_consensus.cc:482] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:38.323101 2232 raft_consensus.cc:504] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "884a11cb30584a1491ac39dfcb1b7ddc" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35923 } } peers { permanent_uuid: "2de7a5e791c74f01b8cfa908b9f2cc8f" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 36009 } } peers { permanent_uuid: "7ae020b2cd044833a31c53b3ec5ad84a" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41637 } }
I20260927 11:11:38.323303 2232 leader_election.cc:288] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923), 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009)
W20260927 11:11:38.323661 1814 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 884a11cb30584a1491ac39dfcb1b7ddc (127.17.95.195:35923): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
W20260927 11:11:38.323767 1813 leader_election.cc:341] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2de7a5e791c74f01b8cfa908b9f2cc8f (127.17.95.194:36009): Not found: Tablet not found: a5021c5e8a524a27baaa9d397ea705a6
I20260927 11:11:38.323843 1813 leader_election.cc:302] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae020b2cd044833a31c53b3ec5ad84a; no voters: 2de7a5e791c74f01b8cfa908b9f2cc8f, 884a11cb30584a1491ac39dfcb1b7ddc
I20260927 11:11:38.323920 2232 raft_consensus.cc:2726] T a5021c5e8a524a27baaa9d397ea705a6 P 7ae020b2cd044833a31c53b3ec5ad84a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 11:11:39.016122 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:39.015883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:37039)
0927 11:11:39.016019 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:39.016057 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260927 11:11:39.063011 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:39.062806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:37525)
0927 11:11:39.062935 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:39.062971 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260927 11:11:39.218791 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:39.218599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:56481)
0927 11:11:39.218725 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:39.218756 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260927 11:11:40.016952 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:40.016735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:41597)
0927 11:11:40.016872 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:40.016908 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":58,"threads_started":1}
I20260927 11:11:40.063596 2237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:40.063437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:33753)
0927 11:11:40.063524 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:40.063555 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":47,"threads_started":1}
I20260927 11:11:40.219456 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:40.219256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:47815)
0927 11:11:40.219382 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:40.219417 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260927 11:11:41.017639 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:41.017415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:41829)
0927 11:11:41.017560 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:41.017596 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260927 11:11:41.064175 2240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:41.063974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:44719)
0927 11:11:41.064095 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:41.064138 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260927 11:11:41.220115 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:41.219880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:47781)
0927 11:11:41.220033 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:41.220073 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260927 11:11:42.018415 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:42.018177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:40457)
0927 11:11:42.018332 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:42.018366 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260927 11:11:42.064929 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:42.064731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:46023)
0927 11:11:42.064863 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:42.064892 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260927 11:11:42.220808 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:42.220564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:38067)
0927 11:11:42.220705 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:42.220759 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260927 11:11:43.019225 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:43.018992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.193:49597)
0927 11:11:43.019140 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:43.019182 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":59,"threads_started":1}
I20260927 11:11:43.065605 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:43.065405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.194:48767)
0927 11:11:43.065521 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:43.065553 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260927 11:11:43.213771 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1762
I20260927 11:11:43.218410 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 1929
I20260927 11:11:43.221532 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:43.221334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44343 (local address 127.17.95.195:55243)
0927 11:11:43.221464 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:43.221495 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.254:44343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260927 11:11:43.223047 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2063
2026-09-27T11:11:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58064 ms)
[----------] 30 tests from SecurityITest (225959 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:43 dist-test-slave-6j0v krb5kdc[2250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:43 dist-test-slave-6j0v krb5kdc[2250](info): set up 2 sockets
Sep 27 11:11:43 dist-test-slave-6j0v krb5kdc[2250](info): commencing operation
krb5kdc: starting...
W20260927 11:11:44.230727 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:11:44.350376 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:42893
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:42893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:44.430563 2263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:44.430796 2263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:44.430835 2263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:44.431967 2263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:44.432030 2263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:44.432055 2263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:44.432085 2263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:44.432114 2263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:44.433532 2263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:42893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:42893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:44.433797 2263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:44.434015 2263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:44.438695 2271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.438709 2269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.438709 2268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:44.438768 2263 server_base.cc:1034] running on GCE node
I20260927 11:11:44.439196 2263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:44.439500 2263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:44.440670 2263 hybrid_clock.cc:648] HybridClock initialized: now 1790507504440670 us; error 40 us; skew 500 ppm
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:44.443351 2263 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:11:44.443832 2263 webserver.cc:460] Webserver started at http://127.17.95.254:40863/ using document root <none> and password file <none>
I20260927 11:11:44.444056 2263 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:44.444099 2263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:44.444231 2263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:44.445259 2263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "f955e906c1884fab95b4f1642e0b2ae9"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "625048fdb8d530579980451f05dbda85"
server_key_iv: "9c778c0bdde4f2404c59fc4348df2b1f"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.445609 2263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "f955e906c1884fab95b4f1642e0b2ae9"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "625048fdb8d530579980451f05dbda85"
server_key_iv: "9c778c0bdde4f2404c59fc4348df2b1f"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.446933 2263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:44.447754 2278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.447921 2263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:44.448006 2263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "f955e906c1884fab95b4f1642e0b2ae9"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "625048fdb8d530579980451f05dbda85"
server_key_iv: "9c778c0bdde4f2404c59fc4348df2b1f"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.448058 2263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:44.458169 2263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:44.459936 2263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:44.460035 2263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:44.463711 2263 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:42893
I20260927 11:11:44.464007 2330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:42893 every 8 connection(s)
I20260927 11:11:44.464048 2263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:44.464665 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2263
I20260927 11:11:44.464752 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:44.464754 2331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:44.464965 17791 external_mini_cluster.cc:1428] Setting key 487a62d792ff1a7db3aa6f352ff1f0af
I20260927 11:11:44.467810 2331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9: Bootstrap starting.
I20260927 11:11:44.468570 2331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:44.468876 2331 log.cc:824] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:44.469551 2331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9: No bootstrap required, opened a new log
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:44.471578 2331 raft_consensus.cc:348] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f955e906c1884fab95b4f1642e0b2ae9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42893 } }
I20260927 11:11:44.471698 2331 raft_consensus.cc:374] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:44.471720 2331 raft_consensus.cc:729] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f955e906c1884fab95b4f1642e0b2ae9, State: Initialized, Role: FOLLOWER
I20260927 11:11:44.471855 2331 consensus_queue.cc:260] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f955e906c1884fab95b4f1642e0b2ae9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42893 } }
I20260927 11:11:44.471933 2331 raft_consensus.cc:388] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:44.471956 2331 raft_consensus.cc:482] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:44.472019 2331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:44.472512 2331 raft_consensus.cc:504] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f955e906c1884fab95b4f1642e0b2ae9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42893 } }
I20260927 11:11:44.472637 2331 leader_election.cc:302] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f955e906c1884fab95b4f1642e0b2ae9; no voters:
I20260927 11:11:44.472790 2331 leader_election.cc:288] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:44.472857 2336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:44.473009 2336 raft_consensus.cc:686] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [term 1 LEADER]: Becoming Leader. State: Replica: f955e906c1884fab95b4f1642e0b2ae9, State: Running, Role: LEADER
I20260927 11:11:44.473130 2331 sys_catalog.cc:564] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:44.473184 2336 consensus_queue.cc:237] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f955e906c1884fab95b4f1642e0b2ae9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42893 } }
I20260927 11:11:44.473513 2336 sys_catalog.cc:455] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f955e906c1884fab95b4f1642e0b2ae9. Latest consensus state: current_term: 1 leader_uuid: "f955e906c1884fab95b4f1642e0b2ae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f955e906c1884fab95b4f1642e0b2ae9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42893 } } }
I20260927 11:11:44.473707 2336 sys_catalog.cc:458] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:44.473523 2338 sys_catalog.cc:455] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f955e906c1884fab95b4f1642e0b2ae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f955e906c1884fab95b4f1642e0b2ae9" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 42893 } } }
I20260927 11:11:44.473973 2338 sys_catalog.cc:458] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:44.474292 2340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:44.474821 2340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:44.476375 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.465439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.17.95.254:42893)
0927 11:11:44.465618 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.465625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.465648 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:11:44.466173 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.466179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.466222 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.466329 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:44.467139 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.468008 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.469147 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.469615 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.469749 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.472316 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:44.472320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:44.472320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:44.472332 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:44.474590 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.475040 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.475044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.475044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.475188 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.475427 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.475428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.475428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.475753 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:44.475919 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.476074 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.476211 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260927 11:11:44.477262 2340 catalog_manager.cc:1347] Generated new cluster ID: da8c7289ff50407face7ea2350783a22
I20260927 11:11:44.477320 2340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:44.495260 2340 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:44.495947 2340 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:44.505265 2340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9: Generated new TSK 0
I20260927 11:11:44.506273 2340 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:11:44.534518 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42893
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:44.613725 2357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:44.613951 2357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:44.613989 2357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:44.615175 2357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:44.615245 2357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:44.615293 2357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:44.616765 2357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:44.617012 2357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:44.617251 2357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:44.622130 2363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.622138 2365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.622151 2362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:44.622181 2357 server_base.cc:1034] running on GCE node
I20260927 11:11:44.622484 2357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:44.622737 2357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:44.623893 2357 hybrid_clock.cc:648] HybridClock initialized: now 1790507504623870 us; error 39 us; skew 500 ppm
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:44.626406 2357 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:11:44.626858 2357 webserver.cc:460] Webserver started at http://127.17.95.193:39027/ using document root <none> and password file <none>
I20260927 11:11:44.627048 2357 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:44.627157 2357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:44.627259 2357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:44.628250 2357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "0306fa8230d640c18d61422806c997ed"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "1060aaa7f8ea9146f769b29bf5791eb1"
server_key_iv: "4f2c25d210a8d052a62aa501656c1699"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.628562 2357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "0306fa8230d640c18d61422806c997ed"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "1060aaa7f8ea9146f769b29bf5791eb1"
server_key_iv: "4f2c25d210a8d052a62aa501656c1699"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.629855 2357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:44.630578 2372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.630744 2357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:44.630815 2357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "0306fa8230d640c18d61422806c997ed"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "1060aaa7f8ea9146f769b29bf5791eb1"
server_key_iv: "4f2c25d210a8d052a62aa501656c1699"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.630873 2357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:44.638831 2357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:44.641006 2357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:44.641105 2357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:44.641283 2357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:44.641513 2357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:44.641541 2357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.641561 2357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:44.641572 2357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.646950 2357 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:37859
I20260927 11:11:44.647055 2485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:37859 every 8 connection(s)
I20260927 11:11:44.647361 2357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:44.649040 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2357
I20260927 11:11:44.649151 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:11:44.649340 17791 external_mini_cluster.cc:1428] Setting key 3a4a808dd2c0bb6cdd4398b1df53349b
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
I20260927 11:11:44.658293 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.648096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:59101 (local address 127.17.95.254:42893)
0927 11:11:44.648164 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.648166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.648339 (+ 173us) server_negotiation.cc:407] Connection header received
0927 11:11:44.648855 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.648857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.648869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.648932 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:44.650042 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.650831 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.652383 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.652590 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.652655 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.655335 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:44.655336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:44.655337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:44.655343 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:44.656784 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.657257 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.657257 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.657258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.657307 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.657592 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.657593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.657593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.657698 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:44.657751 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.658138 (+ 387us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.658229 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:11:44.658504 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.648074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42893 (local address 127.17.95.193:59101)
0927 11:11:44.648221 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:44.648262 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:44.648749 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:44.648994 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:44.648998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:44.649705 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:44.649923 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.649930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.650929 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.650930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.652263 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.652266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.652682 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.652683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.652827 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.653408 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:44.653419 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:44.655241 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:44.656876 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:44.656883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:44.656885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:44.657179 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:44.657374 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:44.657375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:44.657376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:44.657521 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:44.657772 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:44.657779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:44.657934 (+ 155us) client_negotiation.cc:769] Sending connection context
0927 11:11:44.658054 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 11:11:44.658231 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260927 11:11:44.658962 2486 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42893
I20260927 11:11:44.659066 2486 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:44.659260 2486 heartbeater.cc:511] Master 127.17.95.254:42893 requested a full tablet report, sending...
I20260927 11:11:44.659739 2295 ts_manager.cc:194] Registered new tserver with Master: 0306fa8230d640c18d61422806c997ed (127.17.95.193:37859)
I20260927 11:11:44.660466 2295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:59101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:11:44.674222 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42893
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:44.752146 2491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:44.752327 2491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:44.752349 2491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:44.753445 2491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:44.753484 2491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:44.753508 2491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:44.754912 2491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:44.755172 2491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:44.755393 2491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:44.760099 2497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.760248 2496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.760248 2499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:44.760462 2491 server_base.cc:1034] running on GCE node
I20260927 11:11:44.760675 2491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:44.760859 2491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:44.761993 2491 hybrid_clock.cc:648] HybridClock initialized: now 1790507504761984 us; error 28 us; skew 500 ppm
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:44.764389 2491 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:11:44.764801 2491 webserver.cc:460] Webserver started at http://127.17.95.194:36235/ using document root <none> and password file <none>
I20260927 11:11:44.765019 2491 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:44.765084 2491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:44.765234 2491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:44.766228 2491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "0464341bb428447a92e7ea38fd293be3"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "105f868ce944cb06d46332b939ed72d0"
server_key_iv: "45f45319e38290f891891a6f8d667c3f"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.766547 2491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "0464341bb428447a92e7ea38fd293be3"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "105f868ce944cb06d46332b939ed72d0"
server_key_iv: "45f45319e38290f891891a6f8d667c3f"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.767889 2491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:44.768592 2506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.768754 2491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:44.768862 2491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "0464341bb428447a92e7ea38fd293be3"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "105f868ce944cb06d46332b939ed72d0"
server_key_iv: "45f45319e38290f891891a6f8d667c3f"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.768935 2491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:44.783064 2491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:44.784886 2491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:44.785059 2491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:44.785282 2491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:44.785596 2491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:44.785660 2491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.785705 2491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:44.785737 2491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.791973 2491 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:42725
I20260927 11:11:44.792023 2619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:42725 every 8 connection(s)
I20260927 11:11:44.792311 2491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:44.798496 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2491
I20260927 11:11:44.798607 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:11:44.798804 17791 external_mini_cluster.cc:1428] Setting key 3a75aca6c36ee12cfe49189313c758fa
I20260927 11:11:44.801246 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.793047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:53193 (local address 127.17.95.254:42893)
0927 11:11:44.793118 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.793120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.793305 (+ 185us) server_negotiation.cc:407] Connection header received
0927 11:11:44.793748 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.793749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.793758 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.793808 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:44.794582 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.795147 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.796062 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.796271 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.796330 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.798267 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:44.798268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:44.798268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:44.798274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:44.799846 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.800306 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.800308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.800308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.800358 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.800625 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.800626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.800626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.800722 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:44.800805 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.801070 (+ 265us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.801162 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:11:44.801403 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.793037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42893 (local address 127.17.95.194:53193)
0927 11:11:44.793184 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:44.793221 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:44.793669 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:44.793853 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:44.793857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:44.794345 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:44.794506 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.794509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.795218 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.795219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.795976 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.795978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.796342 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.796342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.796432 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.796942 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:44.796949 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:44.798192 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:44.799925 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:44.799932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:44.799934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:44.800225 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:44.800433 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:44.800434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:44.800434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:44.800563 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:44.800810 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:44.800817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:44.800944 (+ 127us) client_negotiation.cc:769] Sending connection context
0927 11:11:44.801047 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 11:11:44.801189 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260927 11:11:44.801846 2620 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42893
I20260927 11:11:44.801924 2620 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:44.802088 2620 heartbeater.cc:511] Master 127.17.95.254:42893 requested a full tablet report, sending...
I20260927 11:11:44.802435 2295 ts_manager.cc:194] Registered new tserver with Master: 0464341bb428447a92e7ea38fd293be3 (127.17.95.194:42725)
I20260927 11:11:44.802948 2295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:53193
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:11:44.821725 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42893
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:44.901012 2625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:44.901199 2625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:44.901221 2625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:44.902321 2625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:44.902354 2625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:44.902376 2625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:44.903774 2625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:44.903945 2625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:44.904102 2625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:44.908841 2633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.908877 2630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:44.908841 2631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:44.908975 2625 server_base.cc:1034] running on GCE node
I20260927 11:11:44.909346 2625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:44.909579 2625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:44.910754 2625 hybrid_clock.cc:648] HybridClock initialized: now 1790507504910740 us; error 42 us; skew 500 ppm
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:44.913280 2625 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:11:44.913757 2625 webserver.cc:460] Webserver started at http://127.17.95.195:35151/ using document root <none> and password file <none>
I20260927 11:11:44.913985 2625 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:44.914050 2625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:44.914206 2625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:44.915323 2625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "a40852b37a024150afd2daea2d6f7241"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "1043b2faceedf3d63be78b0e7ac7ac71"
server_key_iv: "55f4f23c204d9abfdd72143cef5664db"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.915681 2625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "a40852b37a024150afd2daea2d6f7241"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "1043b2faceedf3d63be78b0e7ac7ac71"
server_key_iv: "55f4f23c204d9abfdd72143cef5664db"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.916951 2625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:44.917686 2640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.917850 2625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.917950 2625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "a40852b37a024150afd2daea2d6f7241"
format_stamp: "Formatted at 2026-09-27 11:11:44 on dist-test-slave-6j0v"
server_key: "1043b2faceedf3d63be78b0e7ac7ac71"
server_key_iv: "55f4f23c204d9abfdd72143cef5664db"
server_key_version: "encryptionkey@0"
I20260927 11:11:44.918021 2625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:44.933962 2625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:44.935923 2625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:44.936089 2625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:44.936301 2625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:44.936605 2625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:44.936666 2625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.936707 2625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:44.936746 2625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:44.942535 2625 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:38967
I20260927 11:11:44.942582 2753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:38967 every 8 connection(s)
I20260927 11:11:44.942893 2625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:11:44.946069 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2625
I20260927 11:11:44.946200 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:11:44.946414 17791 external_mini_cluster.cc:1428] Setting key 3a6998d0e4c7d9fc11cda12450ed865b
Sep 27 11:11:44 dist-test-slave-6j0v krb5kdc[2250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:44.952491 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.943594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:38293 (local address 127.17.95.254:42893)
0927 11:11:44.943701 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.943703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.943948 (+ 245us) server_negotiation.cc:407] Connection header received
0927 11:11:44.944472 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.944474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.944487 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.944546 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:44.945479 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.946085 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.947515 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.947767 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.947843 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.949800 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:44.949802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:44.949802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:44.949808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:44.951261 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.951648 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.951649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.951650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.951710 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.951955 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.951956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.951956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.952065 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:44.952113 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.952355 (+ 242us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.952411 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 11:11:44.952732 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.943594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:42893 (local address 127.17.95.195:38293)
0927 11:11:44.943783 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:44.943834 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:44.944358 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:44.944617 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:44.944624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:44.945188 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:44.945374 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.945378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.946193 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.946195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.947441 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.947444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.947856 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.947857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.947958 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.948368 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:44.948376 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:44.949719 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:44.951330 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:44.951336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:44.951337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:44.951552 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:44.951789 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:44.951790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:44.951790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:44.951898 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:44.952117 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:44.952122 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:44.952227 (+ 105us) client_negotiation.cc:769] Sending connection context
0927 11:11:44.952340 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 11:11:44.952473 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":122,"threads_started":1}
I20260927 11:11:44.953231 2754 heartbeater.cc:347] Connected to a master server at 127.17.95.254:42893
I20260927 11:11:44.953321 2754 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:44.953449 2754 heartbeater.cc:511] Master 127.17.95.254:42893 requested a full tablet report, sending...
I20260927 11:11:44.953804 2295 ts_manager.cc:194] Registered new tserver with Master: a40852b37a024150afd2daea2d6f7241 (127.17.95.195:38967)
I20260927 11:11:44.954176 2295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:38293
I20260927 11:11:44.958267 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:44.965571 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.959622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.17.95.254:42893)
0927 11:11:44.959687 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.959689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.959829 (+ 140us) server_negotiation.cc:407] Connection header received
0927 11:11:44.959849 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.959849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.959857 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.959901 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:44.960397 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.960896 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.961618 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.961798 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.961852 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.962916 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:44.962917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:44.962918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:44.962923 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:44.964293 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.964626 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.964627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.964627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.964681 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:44.964931 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:44.964933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:44.964933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:44.965149 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:44.965266 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.965384 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.965468 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:11:44.967368 2295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:44.968966 2295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:44.975980 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.971429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.17.95.193:37859)
0927 11:11:44.971577 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.971581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.972153 (+ 572us) server_negotiation.cc:407] Connection header received
0927 11:11:44.972199 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.972201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.972217 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.972288 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:44.972384 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.973184 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.974805 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.975539 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.975622 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.975672 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.975848 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.975894 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260927 11:11:44.975998 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.971121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:38967 (local address 127.0.0.1:58320)
0927 11:11:44.971718 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:44.971755 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:44.971835 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:44.972090 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:44.972093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:44.972096 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:44.972168 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.972169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.973539 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.973540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.974649 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.974652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.975705 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.975706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.975782 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.975785 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:44.975854 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:11:44.975893 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":3,"thread_start_us":55,"threads_started":1}
I20260927 11:11:44.975994 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.971350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:37859 (local address 127.0.0.1:42546)
0927 11:11:44.972046 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:44.972054 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:44.972093 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:44.972283 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:44.972284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:44.972286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:44.972325 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.972326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.973286 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.973287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.974734 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.974737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.975638 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.975639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.975714 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.975729 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 11:11:44.975817 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 11:11:44.975869 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":4,"thread_start_us":45,"threads_started":1}
I20260927 11:11:44.976464 2764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.971206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.17.95.195:38967)
0927 11:11:44.971455 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.971461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.971823 (+ 362us) server_negotiation.cc:407] Connection header received
0927 11:11:44.971908 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.971910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.971927 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.972001 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:44.972230 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.973385 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.974730 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.975539 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.975611 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.975669 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.976034 (+ 365us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.976084 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
I20260927 11:11:44.976583 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.971477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.17.95.194:42725)
0927 11:11:44.971644 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:44.971647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:44.972002 (+ 355us) server_negotiation.cc:407] Connection header received
0927 11:11:44.972052 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:44.972055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:44.972071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:44.972148 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:44.972378 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.973402 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.975150 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.975860 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.975917 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.975973 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:44.976464 (+ 491us) server_negotiation.cc:299] Negotiation successful
0927 11:11:44.976503 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":122,"threads_started":1}
I20260927 11:11:44.976673 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:44.971129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:42725 (local address 127.0.0.1:42024)
0927 11:11:44.971927 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:44.971935 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:44.971983 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:44.972235 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:44.972236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:44.972238 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:44.972300 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.972301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.973512 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.973512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.974735 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:44.974737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:44.976005 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:44.976006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:44.976065 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:44.976070 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 11:11:44.976132 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 11:11:44.976173 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":3,"thread_start_us":51,"threads_started":1}
I20260927 11:11:44.976986 2420 tablet_service.cc:1467] Processing CreateTablet for tablet 83d7f70ab6b14ecf9243835301fbe9e2 (DEFAULT_TABLE table=test-table [id=8ec9374c6d6c49a199aabda3a216cfec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:44.977165 2688 tablet_service.cc:1467] Processing CreateTablet for tablet 83d7f70ab6b14ecf9243835301fbe9e2 (DEFAULT_TABLE table=test-table [id=8ec9374c6d6c49a199aabda3a216cfec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:44.977339 2420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d7f70ab6b14ecf9243835301fbe9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:44.977255 2554 tablet_service.cc:1467] Processing CreateTablet for tablet 83d7f70ab6b14ecf9243835301fbe9e2 (DEFAULT_TABLE table=test-table [id=8ec9374c6d6c49a199aabda3a216cfec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:44.977491 2688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d7f70ab6b14ecf9243835301fbe9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:44.977545 2554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d7f70ab6b14ecf9243835301fbe9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:44.980170 2768 tablet_bootstrap.cc:492] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Bootstrap starting.
I20260927 11:11:44.980621 2769 tablet_bootstrap.cc:492] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Bootstrap starting.
I20260927 11:11:44.980803 2768 tablet_bootstrap.cc:654] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:44.981331 2769 tablet_bootstrap.cc:654] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:44.981468 2768 log.cc:824] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:44.981892 2769 log.cc:824] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:44.982242 2770 tablet_bootstrap.cc:492] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed: Bootstrap starting.
I20260927 11:11:44.982280 2768 tablet_bootstrap.cc:492] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: No bootstrap required, opened a new log
I20260927 11:11:44.982339 2768 ts_tablet_manager.cc:1397] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:44.982579 2769 tablet_bootstrap.cc:492] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: No bootstrap required, opened a new log
I20260927 11:11:44.982705 2769 ts_tablet_manager.cc:1397] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:44.982897 2770 tablet_bootstrap.cc:654] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:44.983500 2770 log.cc:824] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:44.984306 2770 tablet_bootstrap.cc:492] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed: No bootstrap required, opened a new log
I20260927 11:11:44.984376 2770 ts_tablet_manager.cc:1397] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:44.984525 2769 raft_consensus.cc:348] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:44.984685 2769 raft_consensus.cc:374] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:44.984568 2768 raft_consensus.cc:348] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:44.984746 2769 raft_consensus.cc:729] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0464341bb428447a92e7ea38fd293be3, State: Initialized, Role: FOLLOWER
I20260927 11:11:44.984790 2768 raft_consensus.cc:374] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:44.984841 2768 raft_consensus.cc:729] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40852b37a024150afd2daea2d6f7241, State: Initialized, Role: FOLLOWER
I20260927 11:11:44.984897 2769 consensus_queue.cc:260] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:44.984954 2768 consensus_queue.cc:260] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:44.985188 2769 ts_tablet_manager.cc:1428] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:44.985234 2768 ts_tablet_manager.cc:1428] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:44.985270 2754 heartbeater.cc:503] Master 127.17.95.254:42893 was elected leader, sending a full tablet report...
I20260927 11:11:44.985333 2620 heartbeater.cc:503] Master 127.17.95.254:42893 was elected leader, sending a full tablet report...
I20260927 11:11:44.986471 2770 raft_consensus.cc:348] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:44.986637 2770 raft_consensus.cc:374] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:44.986663 2770 raft_consensus.cc:729] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0306fa8230d640c18d61422806c997ed, State: Initialized, Role: FOLLOWER
I20260927 11:11:44.986776 2770 consensus_queue.cc:260] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:44.987039 2770 ts_tablet_manager.cc:1428] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:44.987154 2486 heartbeater.cc:503] Master 127.17.95.254:42893 was elected leader, sending a full tablet report...
W20260927 11:11:45.043018 2621 tablet.cc:2369] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:11:45.085628 2775 raft_consensus.cc:482] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:45.085822 2775 raft_consensus.cc:504] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:45.086166 2775 leader_election.cc:288] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0464341bb428447a92e7ea38fd293be3 (127.17.95.194:42725), 0306fa8230d640c18d61422806c997ed (127.17.95.193:37859)
I20260927 11:11:45.089134 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.086447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:55279 (local address 127.17.95.194:42725)
0927 11:11:45.086491 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.086493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.086597 (+ 104us) server_negotiation.cc:407] Connection header received
0927 11:11:45.086669 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.086670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.086682 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.086762 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.086924 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.087477 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.088391 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.088769 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.088852 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.088860 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.089016 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.089052 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:11:45.089144 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.086412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:45785 (local address 127.17.95.193:37859)
0927 11:11:45.086480 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.086482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.086493 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:11:45.086532 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.086532 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.086549 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.086637 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.086830 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.087396 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.088343 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.088737 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.088841 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.088852 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.089023 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.089064 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 11:11:45.089150 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.086323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:37859 (local address 127.17.95.195:45785)
0927 11:11:45.086363 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.086377 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.086422 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.086657 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.086658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.086660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.086739 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.086741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.087493 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.087494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.088241 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.088243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.088839 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.088840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.088931 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.088934 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.089015 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.089059 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":3}
I20260927 11:11:45.089164 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.086337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:42725 (local address 127.17.95.195:55279)
0927 11:11:45.086528 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.086545 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.086614 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.086768 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.086768 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.086770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.086849 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.086850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.087558 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.087558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.088302 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.088304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.088851 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.088852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.088933 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.088936 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.089008 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.089048 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260927 11:11:45.089540 2440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d7f70ab6b14ecf9243835301fbe9e2" candidate_uuid: "a40852b37a024150afd2daea2d6f7241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0306fa8230d640c18d61422806c997ed" is_pre_election: true
I20260927 11:11:45.089661 2440 raft_consensus.cc:2454] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a40852b37a024150afd2daea2d6f7241 in term 0.
I20260927 11:11:45.089913 2642 leader_election.cc:302] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0306fa8230d640c18d61422806c997ed, a40852b37a024150afd2daea2d6f7241; no voters:
I20260927 11:11:45.090045 2775 raft_consensus.cc:2781] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:45.090049 2574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d7f70ab6b14ecf9243835301fbe9e2" candidate_uuid: "a40852b37a024150afd2daea2d6f7241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0464341bb428447a92e7ea38fd293be3" is_pre_election: true
I20260927 11:11:45.090119 2775 raft_consensus.cc:482] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:45.090152 2775 raft_consensus.cc:3037] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.090202 2574 raft_consensus.cc:2454] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a40852b37a024150afd2daea2d6f7241 in term 0.
I20260927 11:11:45.090916 2775 raft_consensus.cc:504] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:45.091059 2775 leader_election.cc:288] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [CANDIDATE]: Term 1 election: Requested vote from peers 0464341bb428447a92e7ea38fd293be3 (127.17.95.194:42725), 0306fa8230d640c18d61422806c997ed (127.17.95.193:37859)
I20260927 11:11:45.091218 2440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d7f70ab6b14ecf9243835301fbe9e2" candidate_uuid: "a40852b37a024150afd2daea2d6f7241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0306fa8230d640c18d61422806c997ed"
I20260927 11:11:45.091270 2574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d7f70ab6b14ecf9243835301fbe9e2" candidate_uuid: "a40852b37a024150afd2daea2d6f7241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0464341bb428447a92e7ea38fd293be3"
I20260927 11:11:45.091315 2440 raft_consensus.cc:3037] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.091372 2574 raft_consensus.cc:3037] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.092020 2440 raft_consensus.cc:2454] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a40852b37a024150afd2daea2d6f7241 in term 1.
I20260927 11:11:45.092219 2642 leader_election.cc:302] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0306fa8230d640c18d61422806c997ed, a40852b37a024150afd2daea2d6f7241; no voters:
I20260927 11:11:45.092262 2574 raft_consensus.cc:2454] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a40852b37a024150afd2daea2d6f7241 in term 1.
I20260927 11:11:45.092314 2775 raft_consensus.cc:2781] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:45.092455 2775 raft_consensus.cc:686] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 1 LEADER]: Becoming Leader. State: Replica: a40852b37a024150afd2daea2d6f7241, State: Running, Role: LEADER
I20260927 11:11:45.092545 2775 consensus_queue.cc:237] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } }
I20260927 11:11:45.093415 2294 catalog_manager.cc:5495] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 reported cstate change: term changed from 0 to 1, leader changed from <none> to a40852b37a024150afd2daea2d6f7241 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "a40852b37a024150afd2daea2d6f7241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a40852b37a024150afd2daea2d6f7241" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 38967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 } health_report { overall_health: UNKNOWN } } }
I20260927 11:11:45.118721 2764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.116544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.17.95.195:38967)
0927 11:11:45.116596 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.116599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.116609 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:11:45.116649 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.116650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.116671 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.116725 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:45.116828 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.117303 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.118024 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.118253 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.118323 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.118451 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:45.118547 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:45.118608 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.118631 (+ 23us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.118647 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:11:45.120589 2440 raft_consensus.cc:1261] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0306fa8230d640c18d61422806c997ed [term 1 FOLLOWER]: Refusing update from remote peer a40852b37a024150afd2daea2d6f7241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:45.120611 2574 raft_consensus.cc:1261] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 1 FOLLOWER]: Refusing update from remote peer a40852b37a024150afd2daea2d6f7241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:45.120967 2778 consensus_queue.cc:1035] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0306fa8230d640c18d61422806c997ed" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:45.121158 2775 consensus_queue.cc:1035] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0464341bb428447a92e7ea38fd293be3" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:45.126534 2294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:45.126649 2294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:45.127452 2294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f955e906c1884fab95b4f1642e0b2ae9: Sending DeleteTablet for 3 replicas of tablet 83d7f70ab6b14ecf9243835301fbe9e2
I20260927 11:11:45.127815 2688 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d7f70ab6b14ecf9243835301fbe9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:45 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:58320
I20260927 11:11:45.127863 2554 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d7f70ab6b14ecf9243835301fbe9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:45 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:42024
I20260927 11:11:45.128027 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2357
I20260927 11:11:45.128059 2420 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d7f70ab6b14ecf9243835301fbe9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:45 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:42546
I20260927 11:11:45.128136 2791 tablet_replica.cc:330] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: stopping tablet replica
I20260927 11:11:45.128223 2791 raft_consensus.cc:2229] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:11:45.128326 2791 raft_consensus.cc:2258] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:45.128453 2792 tablet_replica.cc:330] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: stopping tablet replica
I20260927 11:11:45.128525 2792 raft_consensus.cc:2229] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:45.128610 2792 raft_consensus.cc:2258] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:45.129087 2791 ts_tablet_manager.cc:1905] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:45.130399 2792 ts_tablet_manager.cc:1905] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:45.130596 2791 ts_tablet_manager.cc:1918] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:45.130659 2791 log.cc:1196] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/83d7f70ab6b14ecf9243835301fbe9e2
I20260927 11:11:45.130944 2791 ts_tablet_manager.cc:1939] T 83d7f70ab6b14ecf9243835301fbe9e2 P a40852b37a024150afd2daea2d6f7241: Deleting consensus metadata
I20260927 11:11:45.131364 2280 catalog_manager.cc:4841] TS a40852b37a024150afd2daea2d6f7241 (127.17.95.195:38967): tablet 83d7f70ab6b14ecf9243835301fbe9e2 (table test-table [id=8ec9374c6d6c49a199aabda3a216cfec]) successfully deleted
I20260927 11:11:45.132040 2792 ts_tablet_manager.cc:1918] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:45.132089 2792 log.cc:1196] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/83d7f70ab6b14ecf9243835301fbe9e2
I20260927 11:11:45.132434 2792 ts_tablet_manager.cc:1939] T 83d7f70ab6b14ecf9243835301fbe9e2 P 0464341bb428447a92e7ea38fd293be3: Deleting consensus metadata
I20260927 11:11:45.132849 2282 catalog_manager.cc:4841] TS 0464341bb428447a92e7ea38fd293be3 (127.17.95.194:42725): tablet 83d7f70ab6b14ecf9243835301fbe9e2 (table test-table [id=8ec9374c6d6c49a199aabda3a216cfec]) successfully deleted
W20260927 11:11:45.136978 2280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:37859 (error 108)
I20260927 11:11:45.137266 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2491
I20260927 11:11:45.137492 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.137368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:37859 (local address 127.0.0.1:42554)
0927 11:11:45.137419 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.137452 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:37859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 11:11:45.137677 2280 catalog_manager.cc:4584] TS 0306fa8230d640c18d61422806c997ed (127.17.95.193:37859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83d7f70ab6b14ecf9243835301fbe9e2: Network error: Client connection negotiation failed: client connection to 127.17.95.193:37859: connect: Connection refused (error 111)
I20260927 11:11:45.143059 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2625
I20260927 11:11:45.148403 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2263
2026-09-27T11:11:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1925 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T11:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:45Z Disabled control of system clock
I20260927 11:11:45.176476 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:41287
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:41287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:45.258708 2797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:45.258919 2797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:45.258941 2797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:45.260171 2797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:45.260215 2797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:45.260231 2797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:45.260245 2797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:45.260259 2797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:45.261545 2797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:41287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:41287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:45.261742 2797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:45.261989 2797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:45.266535 2802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:45.266551 2803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:45.266546 2805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:45.266757 2797 server_base.cc:1034] running on GCE node
I20260927 11:11:45.266981 2797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:45.267267 2797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:45.268468 2797 hybrid_clock.cc:648] HybridClock initialized: now 1790507505268448 us; error 42 us; skew 500 ppm
I20260927 11:11:45.269731 2797 webserver.cc:460] Webserver started at http://127.17.95.254:34223/ using document root <none> and password file <none>
I20260927 11:11:45.269990 2797 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:45.270061 2797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:45.270221 2797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:45.271281 2797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "2449eb87d0d149089b1fa4442795951c"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "7e115be2f2d7eeb9830c4700a4736563"
server_key_iv: "ca4c3d5099263ab5ed57b95f18a295d0"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.271667 2797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "2449eb87d0d149089b1fa4442795951c"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "7e115be2f2d7eeb9830c4700a4736563"
server_key_iv: "ca4c3d5099263ab5ed57b95f18a295d0"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.272950 2797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:45.273710 2811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.273878 2797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:45.273974 2797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "2449eb87d0d149089b1fa4442795951c"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "7e115be2f2d7eeb9830c4700a4736563"
server_key_iv: "ca4c3d5099263ab5ed57b95f18a295d0"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.274050 2797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:45.284339 2797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:45.284621 2797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:45.284770 2797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:45.288764 2797 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:41287
I20260927 11:11:45.288800 2863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:41287 every 8 connection(s)
I20260927 11:11:45.289105 2797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:45.289991 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:45.290555 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2797
I20260927 11:11:45.290719 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:45.290918 17791 external_mini_cluster.cc:1428] Setting key 543b71c8d8fdc493a9266d2a8e594f49
I20260927 11:11:45.292723 2864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: Bootstrap starting.
I20260927 11:11:45.293257 2864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:45.293522 2864 log.cc:824] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:45.294221 2864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: No bootstrap required, opened a new log
I20260927 11:11:45.296607 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.291436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.17.95.254:41287)
0927 11:11:45.291668 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.291675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.291699 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:11:45.292094 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.292100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.292153 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.292311 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:45.292575 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.293530 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.294639 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.295051 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.295273 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.295930 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:45.295933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:45.295933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:45.296094 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:45.296126 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:45.296137 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:45.296191 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.296297 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.296353 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260927 11:11:45.296564 2864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2449eb87d0d149089b1fa4442795951c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41287 } }
I20260927 11:11:45.296675 2864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:45.296726 2864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2449eb87d0d149089b1fa4442795951c, State: Initialized, Role: FOLLOWER
I20260927 11:11:45.296844 2864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2449eb87d0d149089b1fa4442795951c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41287 } }
I20260927 11:11:45.296923 2864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:45.296953 2864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:45.296996 2864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.297539 2864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2449eb87d0d149089b1fa4442795951c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41287 } }
I20260927 11:11:45.297624 2864 leader_election.cc:302] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2449eb87d0d149089b1fa4442795951c; no voters:
I20260927 11:11:45.297827 2864 leader_election.cc:288] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:45.297907 2869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:45.298055 2869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [term 1 LEADER]: Becoming Leader. State: Replica: 2449eb87d0d149089b1fa4442795951c, State: Running, Role: LEADER
I20260927 11:11:45.298090 2864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:45.298146 2869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2449eb87d0d149089b1fa4442795951c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41287 } }
I20260927 11:11:45.298542 2871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2449eb87d0d149089b1fa4442795951c. Latest consensus state: current_term: 1 leader_uuid: "2449eb87d0d149089b1fa4442795951c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2449eb87d0d149089b1fa4442795951c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41287 } } }
I20260927 11:11:45.298643 2871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:45.299116 2870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2449eb87d0d149089b1fa4442795951c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2449eb87d0d149089b1fa4442795951c" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 41287 } } }
I20260927 11:11:45.299214 2870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c [sys.catalog]: This master's current role is: LEADER
W20260927 11:11:45.299865 2885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:11:45.299929 2885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:11:45.299978 2878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:45.300149 2878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:45.301630 2878 catalog_manager.cc:1347] Generated new cluster ID: f5b3f7b2514c42e5b048dd003b2c3ddd
I20260927 11:11:45.301718 2878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:45.321448 2878 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:45.321952 2878 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:45.326380 2878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: Generated new TSK 0
I20260927 11:11:45.326565 2878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:45.335541 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:41287
--builtin_ntp_servers=127.17.95.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:45.415611 2888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:45.415808 2888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:45.415831 2888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:45.416987 2888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:45.417032 2888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:45.417059 2888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:45.418428 2888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:45.418672 2888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:45.418955 2888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:45.423734 2896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:45.423745 2893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:45.423754 2894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:45.424062 2888 server_base.cc:1034] running on GCE node
I20260927 11:11:45.424269 2888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:45.424561 2888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:45.425727 2888 hybrid_clock.cc:648] HybridClock initialized: now 1790507505425716 us; error 29 us; skew 500 ppm
I20260927 11:11:45.426911 2888 webserver.cc:460] Webserver started at http://127.17.95.193:34397/ using document root <none> and password file <none>
I20260927 11:11:45.427163 2888 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:45.427220 2888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:45.427310 2888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:45.428345 2888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "72548c1eed944133bcb81ac279a5d225"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "e07bc178b1c6589ba7c2009fddb2bca8"
server_key_iv: "357a2e603fc5b69cd3b6f54bb85716ef"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.428683 2888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "72548c1eed944133bcb81ac279a5d225"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "e07bc178b1c6589ba7c2009fddb2bca8"
server_key_iv: "357a2e603fc5b69cd3b6f54bb85716ef"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.429829 2888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:11:45.430491 2902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.430666 2888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:11:45.430747 2888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "72548c1eed944133bcb81ac279a5d225"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "e07bc178b1c6589ba7c2009fddb2bca8"
server_key_iv: "357a2e603fc5b69cd3b6f54bb85716ef"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.430827 2888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:45.459828 2888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:45.460126 2888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:45.460260 2888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:45.460479 2888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:45.460788 2888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:45.460824 2888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.460845 2888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:45.460888 2888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.466378 2888 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:35917
I20260927 11:11:45.466454 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:35917 every 8 connection(s)
I20260927 11:11:45.466748 2888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:45.470448 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 2888
I20260927 11:11:45.470541 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:11:45.470748 17791 external_mini_cluster.cc:1428] Setting key ca51eb529bec72b18de82ab5f7989682
I20260927 11:11:45.471733 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:41287
--builtin_ntp_servers=127.17.95.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 11:11:45.472308 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.467548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:51631 (local address 127.17.95.254:41287)
0927 11:11:45.467592 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.467594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.467770 (+ 176us) server_negotiation.cc:407] Connection header received
0927 11:11:45.468270 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.468271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.468281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.468339 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:45.469036 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.469525 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.470708 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.470998 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.471066 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.472006 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:45.472008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:45.472008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:45.472014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:45.472031 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:45.472035 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:45.472092 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.472165 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.472197 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:11:45.472411 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.467459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:41287 (local address 127.17.95.193:51631)
0927 11:11:45.467661 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.467700 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.468188 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.468390 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.468393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.468401 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:45.468939 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.468944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.469606 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.469606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.470616 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.470628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.471104 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.471105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.471248 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.471797 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:45.471797 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:45.471857 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:45.471883 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:45.472098 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:45.472102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:45.472105 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.472165 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.472239 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260927 11:11:45.472863 3016 heartbeater.cc:347] Connected to a master server at 127.17.95.254:41287
I20260927 11:11:45.472960 3016 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:45.473153 3016 heartbeater.cc:511] Master 127.17.95.254:41287 requested a full tablet report, sending...
I20260927 11:11:45.473713 2828 ts_manager.cc:194] Registered new tserver with Master: 72548c1eed944133bcb81ac279a5d225 (127.17.95.193:35917)
I20260927 11:11:45.474315 2828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.193:51631
W20260927 11:11:45.552107 3019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:45.552295 3019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:45.552316 3019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:45.553440 3019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:45.553478 3019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:45.553501 3019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:45.554790 3019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:45.555006 3019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:45.555290 3019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:45.560051 3024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:45.560118 3019 server_base.cc:1034] running on GCE node
W20260927 11:11:45.560101 3025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:45.560077 3027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:45.560452 3019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:45.560704 3019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:45.561867 3019 hybrid_clock.cc:648] HybridClock initialized: now 1790507505561860 us; error 45 us; skew 500 ppm
I20260927 11:11:45.563184 3019 webserver.cc:460] Webserver started at http://127.17.95.194:40643/ using document root <none> and password file <none>
I20260927 11:11:45.563416 3019 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:45.563482 3019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:45.563616 3019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:45.564634 3019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "3ef6c5f17921446094b6029a82f19a00"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "9555b1f20f80c9701348d13eced201c6"
server_key_iv: "2141335b84c8e700f6fba8940daa6817"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.564980 3019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "3ef6c5f17921446094b6029a82f19a00"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "9555b1f20f80c9701348d13eced201c6"
server_key_iv: "2141335b84c8e700f6fba8940daa6817"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.566249 3019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:45.567008 3033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.567234 3019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:45.567345 3019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "3ef6c5f17921446094b6029a82f19a00"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "9555b1f20f80c9701348d13eced201c6"
server_key_iv: "2141335b84c8e700f6fba8940daa6817"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.567435 3019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:45.581156 3019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:45.581429 3019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:45.581575 3019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:45.581795 3019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:45.582098 3019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:45.582155 3019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.582190 3019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:45.582216 3019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.587920 3019 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:33359
I20260927 11:11:45.587958 3146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:33359 every 8 connection(s)
I20260927 11:11:45.588295 3019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:11:45.593631 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.589029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:39723 (local address 127.17.95.254:41287)
0927 11:11:45.589093 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.589095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.589324 (+ 229us) server_negotiation.cc:407] Connection header received
0927 11:11:45.589987 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.589988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.590014 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.590071 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:45.590740 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.591288 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.592220 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.592418 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.592500 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.593360 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:45.593361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:45.593361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:45.593366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:45.593381 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:45.593385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:45.593429 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.593506 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.593532 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:11:45.593837 3149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.589029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:41287 (local address 127.17.95.194:39723)
0927 11:11:45.589205 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.589237 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.589888 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.590163 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.590166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.590176 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:45.590646 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.590650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.591370 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.591371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.592097 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.592099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.592510 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.592511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.592676 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.593219 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:45.593219 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:45.593246 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:45.593267 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:45.593439 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:45.593443 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:45.593447 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.593508 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.593615 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260927 11:11:45.594313 3147 heartbeater.cc:347] Connected to a master server at 127.17.95.254:41287
I20260927 11:11:45.594400 3147 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:45.594532 3147 heartbeater.cc:511] Master 127.17.95.254:41287 requested a full tablet report, sending...
I20260927 11:11:45.594946 2828 ts_manager.cc:194] Registered new tserver with Master: 3ef6c5f17921446094b6029a82f19a00 (127.17.95.194:33359)
I20260927 11:11:45.595335 2828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.194:39723
I20260927 11:11:45.596915 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3019
I20260927 11:11:45.596990 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:11:45.597133 17791 external_mini_cluster.cc:1428] Setting key bf7f9bd825aae35a3962fb14e4f82bec
I20260927 11:11:45.598093 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:41287
--builtin_ntp_servers=127.17.95.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:45.677656 3150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:45.677845 3150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:45.677863 3150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:45.679001 3150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:45.679039 3150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:45.679064 3150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:45.680372 3150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:45.680545 3150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:45.680770 3150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:45.685516 3158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:45.685524 3156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:45.685698 3150 server_base.cc:1034] running on GCE node
W20260927 11:11:45.685597 3155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:45.686033 3150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:45.686295 3150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:45.687467 3150 hybrid_clock.cc:648] HybridClock initialized: now 1790507505687459 us; error 45 us; skew 500 ppm
I20260927 11:11:45.688853 3150 webserver.cc:460] Webserver started at http://127.17.95.195:42299/ using document root <none> and password file <none>
I20260927 11:11:45.689128 3150 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:45.689198 3150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:45.689354 3150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:45.690383 3150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "ef55a30711274e59b438c4c3dc3fdb88"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "011828f227e0354347780800e085ced7"
server_key_iv: "5eec2de1d1b17350ea2b4829426c322b"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.690764 3150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "ef55a30711274e59b438c4c3dc3fdb88"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "011828f227e0354347780800e085ced7"
server_key_iv: "5eec2de1d1b17350ea2b4829426c322b"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.692210 3150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:11:45.693006 3164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.693198 3150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:11:45.693305 3150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "ef55a30711274e59b438c4c3dc3fdb88"
format_stamp: "Formatted at 2026-09-27 11:11:45 on dist-test-slave-6j0v"
server_key: "011828f227e0354347780800e085ced7"
server_key_iv: "5eec2de1d1b17350ea2b4829426c322b"
server_key_version: "encryptionkey@0"
I20260927 11:11:45.693387 3150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:45.714700 3150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:45.715009 3150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:45.715183 3150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:45.715404 3150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:45.715734 3150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:45.715787 3150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.715829 3150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:45.715857 3150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:45.721686 3150 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:36913
I20260927 11:11:45.721724 3277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:36913 every 8 connection(s)
I20260927 11:11:45.722054 3150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:11:45.722918 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3150
I20260927 11:11:45.723031 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:11:45.723248 17791 external_mini_cluster.cc:1428] Setting key 2b3202d80dca1f696d52222acaafe4fd
I20260927 11:11:45.727584 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.722824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:57867 (local address 127.17.95.254:41287)
0927 11:11:45.722906 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.722911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.723103 (+ 192us) server_negotiation.cc:407] Connection header received
0927 11:11:45.723680 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.723681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.723693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.723750 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:45.724429 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.725058 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.726117 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.726325 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.726420 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.727270 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:45.727271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:45.727272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:45.727276 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:45.727289 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:45.727293 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:45.727388 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.727482 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.727507 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:11:45.727837 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.722777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:41287 (local address 127.17.95.195:57867)
0927 11:11:45.722957 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.723002 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.723569 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.723832 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.723836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.723855 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:45.724341 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.724345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.725165 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.725166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.726019 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.726022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.726445 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.726446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.726576 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.727127 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:45.727128 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:45.727155 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:45.727176 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:45.727393 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:45.727398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:45.727401 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.727485 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.727584 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260927 11:11:45.728277 3278 heartbeater.cc:347] Connected to a master server at 127.17.95.254:41287
I20260927 11:11:45.728361 3278 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:45.728513 3278 heartbeater.cc:511] Master 127.17.95.254:41287 requested a full tablet report, sending...
I20260927 11:11:45.728900 2828 ts_manager.cc:194] Registered new tserver with Master: ef55a30711274e59b438c4c3dc3fdb88 (127.17.95.195:36913)
I20260927 11:11:45.729277 2828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.195:57867
I20260927 11:11:45.734836 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:45.738885 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.736314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.17.95.254:41287)
0927 11:11:45.736369 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.736371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.736436 (+ 65us) server_negotiation.cc:407] Connection header received
0927 11:11:45.736480 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.736480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.736488 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.736537 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:45.736700 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.737136 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.737842 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.738007 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.738054 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.738665 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:45.738666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:45.738666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:45.738670 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:45.738681 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:45.738685 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:45.738727 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.738820 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.738837 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:11:45.740298 2828 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:45.741806 2828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:45.747915 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.743863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35917 (local address 127.0.0.1:43552)
0927 11:11:45.744108 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.744147 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.744210 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.744454 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.744456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.744458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.744524 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.744526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.745918 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.745919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.746777 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.746780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.747624 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.747698 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.747710 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.747780 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.747830 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":2,"thread_start_us":59,"threads_started":1}
I20260927 11:11:45.747926 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.743937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.17.95.193:35917)
0927 11:11:45.744280 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.744283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.744293 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:11:45.744335 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.744336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.744370 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.744459 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.744764 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.745828 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.746838 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.747526 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747600 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.747686 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.747803 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.747849 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":76,"threads_started":1}
I20260927 11:11:45.748101 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.744068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.17.95.195:36913)
0927 11:11:45.744222 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.744226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.744403 (+ 177us) server_negotiation.cc:407] Connection header received
0927 11:11:45.744439 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.744441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.744454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.744522 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.744653 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.745788 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747114 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.747603 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747668 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.747750 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.747951 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.747995 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260927 11:11:45.748286 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.743883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:33359 (local address 127.0.0.1:60278)
0927 11:11:45.744136 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.744147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.744210 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.744825 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.744825 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.744828 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.744870 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.744871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.745706 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.745707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.746992 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.746994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.748026 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.748027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.748088 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.748091 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.748171 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.748210 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":62,"thread_start_us":74,"threads_started":1}
I20260927 11:11:45.748302 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.744084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.17.95.194:33359)
0927 11:11:45.744586 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.744590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.744599 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:11:45.744632 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.744634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.744647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.744758 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.744943 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.745617 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747089 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.747812 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747873 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.747947 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.748190 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.748224 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":84,"threads_started":1}
I20260927 11:11:45.748476 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.743883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:36913 (local address 127.0.0.1:52834)
0927 11:11:45.744302 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.744309 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.744356 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.744534 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.744535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.744536 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.744568 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.744569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.745876 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.745877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.746985 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.746987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.747782 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.747784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.747831 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.747834 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.747878 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.747907 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":48,"thread_start_us":73,"threads_started":1}
I20260927 11:11:45.748967 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 9173bbb7e87f4db294508b182f046100 (DEFAULT_TABLE table=test-table [id=d390cae19295401281ba8f0ac84728c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:45.749185 3081 tablet_service.cc:1467] Processing CreateTablet for tablet 9173bbb7e87f4db294508b182f046100 (DEFAULT_TABLE table=test-table [id=d390cae19295401281ba8f0ac84728c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:45.749194 3212 tablet_service.cc:1467] Processing CreateTablet for tablet 9173bbb7e87f4db294508b182f046100 (DEFAULT_TABLE table=test-table [id=d390cae19295401281ba8f0ac84728c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:45.749297 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9173bbb7e87f4db294508b182f046100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:45.749500 3081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9173bbb7e87f4db294508b182f046100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:45.749500 3212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9173bbb7e87f4db294508b182f046100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:45.751830 3292 tablet_bootstrap.cc:492] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225: Bootstrap starting.
I20260927 11:11:45.752425 3292 tablet_bootstrap.cc:654] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:45.752669 3293 tablet_bootstrap.cc:492] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Bootstrap starting.
I20260927 11:11:45.752775 3292 log.cc:824] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:45.753328 3293 tablet_bootstrap.cc:654] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:45.753522 3292 tablet_bootstrap.cc:492] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225: No bootstrap required, opened a new log
I20260927 11:11:45.753640 3292 ts_tablet_manager.cc:1397] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:11:45.753629 3293 log.cc:824] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:45.753943 3294 tablet_bootstrap.cc:492] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Bootstrap starting.
I20260927 11:11:45.754248 3293 tablet_bootstrap.cc:492] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: No bootstrap required, opened a new log
I20260927 11:11:45.754325 3293 ts_tablet_manager.cc:1397] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:11:45.754833 3294 tablet_bootstrap.cc:654] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:45.755189 3294 log.cc:824] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:45.756127 3294 tablet_bootstrap.cc:492] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: No bootstrap required, opened a new log
I20260927 11:11:45.756229 3294 ts_tablet_manager.cc:1397] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:45.756664 3292 raft_consensus.cc:348] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.756920 3292 raft_consensus.cc:374] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:45.757021 3292 raft_consensus.cc:729] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72548c1eed944133bcb81ac279a5d225, State: Initialized, Role: FOLLOWER
I20260927 11:11:45.756975 3293 raft_consensus.cc:348] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.757162 3293 raft_consensus.cc:374] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:45.757210 3293 raft_consensus.cc:729] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef55a30711274e59b438c4c3dc3fdb88, State: Initialized, Role: FOLLOWER
I20260927 11:11:45.757172 3292 consensus_queue.cc:260] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.757344 3293 consensus_queue.cc:260] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.757426 3292 ts_tablet_manager.cc:1428] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 11:11:45.757452 3016 heartbeater.cc:503] Master 127.17.95.254:41287 was elected leader, sending a full tablet report...
I20260927 11:11:45.757630 3293 ts_tablet_manager.cc:1428] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:45.757685 3278 heartbeater.cc:503] Master 127.17.95.254:41287 was elected leader, sending a full tablet report...
I20260927 11:11:45.758320 3294 raft_consensus.cc:348] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.758524 3294 raft_consensus.cc:374] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:45.758580 3294 raft_consensus.cc:729] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef6c5f17921446094b6029a82f19a00, State: Initialized, Role: FOLLOWER
I20260927 11:11:45.758751 3294 consensus_queue.cc:260] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.758849 3300 raft_consensus.cc:482] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:45.758945 3300 raft_consensus.cc:504] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.759298 3294 ts_tablet_manager.cc:1428] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:45.759348 3147 heartbeater.cc:503] Master 127.17.95.254:41287 was elected leader, sending a full tablet report...
I20260927 11:11:45.759658 3300 leader_election.cc:288] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72548c1eed944133bcb81ac279a5d225 (127.17.95.193:35917), ef55a30711274e59b438c4c3dc3fdb88 (127.17.95.195:36913)
I20260927 11:11:45.761942 3149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.759476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35917 (local address 127.17.95.194:41211)
0927 11:11:45.759537 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.759548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.759594 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.759822 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.759823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.759824 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.759883 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.759904 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.760483 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.760484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.761177 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.761179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.761695 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.761696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.761761 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.761764 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.761842 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.761882 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 11:11:45.761940 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.759502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:41211 (local address 127.17.95.193:35917)
0927 11:11:45.759540 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.759542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.759580 (+ 38us) server_negotiation.cc:407] Connection header received
0927 11:11:45.759717 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.759719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.759729 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.759780 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.759985 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.760393 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.761275 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.761621 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.761697 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.761704 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.761850 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.761883 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:11:45.762101 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.759666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:43291 (local address 127.17.95.195:36913)
0927 11:11:45.759703 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.759705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.759713 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:11:45.759734 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.759735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.759742 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.759794 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:45.759922 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.760552 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.761431 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.761776 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.761853 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.761860 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.762014 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.762047 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:11:45.762125 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.759481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:36913 (local address 127.17.95.194:43291)
0927 11:11:45.759637 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.759647 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:45.759694 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:45.759798 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:45.759799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:45.759800 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:45.759863 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.759864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.760626 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.760627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.761331 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:45.761332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.761852 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.761853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:45.761923 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.761926 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:45.762006 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:11:45.762043 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":6,"thread_start_us":84,"threads_started":1}
I20260927 11:11:45.762256 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9173bbb7e87f4db294508b182f046100" candidate_uuid: "3ef6c5f17921446094b6029a82f19a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72548c1eed944133bcb81ac279a5d225" is_pre_election: true
I20260927 11:11:45.762379 2970 raft_consensus.cc:2454] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef6c5f17921446094b6029a82f19a00 in term 0.
I20260927 11:11:45.762585 3034 leader_election.cc:302] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef6c5f17921446094b6029a82f19a00, 72548c1eed944133bcb81ac279a5d225; no voters:
I20260927 11:11:45.762686 3232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9173bbb7e87f4db294508b182f046100" candidate_uuid: "3ef6c5f17921446094b6029a82f19a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef55a30711274e59b438c4c3dc3fdb88" is_pre_election: true
I20260927 11:11:45.762710 3300 raft_consensus.cc:2781] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:45.762758 3300 raft_consensus.cc:482] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:45.762805 3300 raft_consensus.cc:3037] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.762796 3232 raft_consensus.cc:2454] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef6c5f17921446094b6029a82f19a00 in term 0.
I20260927 11:11:45.763477 3300 raft_consensus.cc:504] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.763602 3300 leader_election.cc:288] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [CANDIDATE]: Term 1 election: Requested vote from peers 72548c1eed944133bcb81ac279a5d225 (127.17.95.193:35917), ef55a30711274e59b438c4c3dc3fdb88 (127.17.95.195:36913)
I20260927 11:11:45.763717 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9173bbb7e87f4db294508b182f046100" candidate_uuid: "3ef6c5f17921446094b6029a82f19a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72548c1eed944133bcb81ac279a5d225"
I20260927 11:11:45.763788 3232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9173bbb7e87f4db294508b182f046100" candidate_uuid: "3ef6c5f17921446094b6029a82f19a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef55a30711274e59b438c4c3dc3fdb88"
I20260927 11:11:45.763823 2970 raft_consensus.cc:3037] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.763870 3232 raft_consensus.cc:3037] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:45.764492 3232 raft_consensus.cc:2454] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef6c5f17921446094b6029a82f19a00 in term 1.
I20260927 11:11:45.764532 2970 raft_consensus.cc:2454] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef6c5f17921446094b6029a82f19a00 in term 1.
I20260927 11:11:45.764710 3037 leader_election.cc:302] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3ef6c5f17921446094b6029a82f19a00, 72548c1eed944133bcb81ac279a5d225, ef55a30711274e59b438c4c3dc3fdb88; no voters:
I20260927 11:11:45.764818 3300 raft_consensus.cc:2781] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:45.764899 3300 raft_consensus.cc:686] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef6c5f17921446094b6029a82f19a00, State: Running, Role: LEADER
I20260927 11:11:45.764971 3300 consensus_queue.cc:237] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } }
I20260927 11:11:45.765731 2827 catalog_manager.cc:5495] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef6c5f17921446094b6029a82f19a00 (127.17.95.194). New cstate: current_term: 1 leader_uuid: "3ef6c5f17921446094b6029a82f19a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef6c5f17921446094b6029a82f19a00" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 33359 } health_report { overall_health: HEALTHY } } }
I20260927 11:11:45.772620 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.770253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.17.95.194:33359)
0927 11:11:45.770293 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:45.770296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:45.770305 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:11:45.770365 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:45.770365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:45.770379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:45.770446 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:45.770553 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.771128 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.771868 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:45.772064 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:45.772147 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:45.772303 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:45.772397 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:45.772485 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:45.772503 (+ 18us) server_negotiation.cc:299] Negotiation successful
0927 11:11:45.772520 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:11:45.774278 2970 raft_consensus.cc:1261] T 9173bbb7e87f4db294508b182f046100 P 72548c1eed944133bcb81ac279a5d225 [term 1 FOLLOWER]: Refusing update from remote peer 3ef6c5f17921446094b6029a82f19a00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:45.774298 3232 raft_consensus.cc:1261] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 1 FOLLOWER]: Refusing update from remote peer 3ef6c5f17921446094b6029a82f19a00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:45.774610 3301 consensus_queue.cc:1035] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72548c1eed944133bcb81ac279a5d225" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 35917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:45.774739 3300 consensus_queue.cc:1035] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef55a30711274e59b438c4c3dc3fdb88" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:45.777832 3307 mvcc.cc:205] Tried to move back new op lower bound from 7333918743648825344 to 7333918743614124032. Current Snapshot: MvccSnapshot[applied={T|T < 7333918743648825344}]
I20260927 11:11:45.779831 2828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:45.779927 2828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:45.780653 2828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2449eb87d0d149089b1fa4442795951c: Sending DeleteTablet for 3 replicas of tablet 9173bbb7e87f4db294508b182f046100
I20260927 11:11:45.781018 3081 tablet_service.cc:1514] Processing DeleteTablet for tablet 9173bbb7e87f4db294508b182f046100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:45 UTC) from {username='slave'} at 127.0.0.1:60278
I20260927 11:11:45.781025 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 9173bbb7e87f4db294508b182f046100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:45 UTC) from {username='slave'} at 127.0.0.1:43552
I20260927 11:11:45.781250 3212 tablet_service.cc:1514] Processing DeleteTablet for tablet 9173bbb7e87f4db294508b182f046100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:45 UTC) from {username='slave'} at 127.0.0.1:52834
I20260927 11:11:45.781358 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2888
I20260927 11:11:45.782075 3315 tablet_replica.cc:330] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: stopping tablet replica
I20260927 11:11:45.782183 3315 raft_consensus.cc:2229] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:11:45.782295 3315 raft_consensus.cc:2258] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:45.782465 3316 tablet_replica.cc:330] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: stopping tablet replica
I20260927 11:11:45.782544 3316 raft_consensus.cc:2229] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:45.782632 3316 raft_consensus.cc:2258] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:45.783231 3315 ts_tablet_manager.cc:1905] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:45.784624 3315 ts_tablet_manager.cc:1918] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:45.784718 3315 log.cc:1196] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/9173bbb7e87f4db294508b182f046100
I20260927 11:11:45.784960 3315 ts_tablet_manager.cc:1939] T 9173bbb7e87f4db294508b182f046100 P 3ef6c5f17921446094b6029a82f19a00: Deleting consensus metadata
I20260927 11:11:45.785327 2814 catalog_manager.cc:4841] TS 3ef6c5f17921446094b6029a82f19a00 (127.17.95.194:33359): tablet 9173bbb7e87f4db294508b182f046100 (table test-table [id=d390cae19295401281ba8f0ac84728c7]) successfully deleted
I20260927 11:11:45.787555 3316 ts_tablet_manager.cc:1905] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 11:11:45.788340 2812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:35917 (error 108)
I20260927 11:11:45.788753 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.788632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:35917 (local address 127.0.0.1:43562)
0927 11:11:45.788684 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.788718 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:35917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260927 11:11:45.788858 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3019
W20260927 11:11:45.788857 2812 catalog_manager.cc:4584] TS 72548c1eed944133bcb81ac279a5d225 (127.17.95.193:35917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9173bbb7e87f4db294508b182f046100: Network error: Client connection negotiation failed: client connection to 127.17.95.193:35917: connect: Connection refused (error 111)
I20260927 11:11:45.794055 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3150
I20260927 11:11:45.794066 3316 ts_tablet_manager.cc:1918] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:45.794140 3316 log.cc:1196] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/9173bbb7e87f4db294508b182f046100
I20260927 11:11:45.794368 3316 ts_tablet_manager.cc:1939] T 9173bbb7e87f4db294508b182f046100 P ef55a30711274e59b438c4c3dc3fdb88: Deleting consensus metadata
I20260927 11:11:45.799253 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 2797
I20260927 11:11:45.799330 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:45.799187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:36913 (local address 127.0.0.1:52836)
0927 11:11:45.799233 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:45.799257 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.195:36913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
2026-09-27T11:11:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (649 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:45 dist-test-slave-6j0v krb5kdc[3320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:45 dist-test-slave-6j0v krb5kdc[3320](info): set up 2 sockets
Sep 27 11:11:45 dist-test-slave-6j0v krb5kdc[3320](info): commencing operation
krb5kdc: starting...
W20260927 11:11:46.842805 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:46 dist-test-slave-6j0v krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:11:46.954272 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44627
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:44627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:47.032181 3333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:47.032387 3333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:47.032421 3333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:47.033437 3333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:47.033500 3333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:47.033522 3333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:47.033550 3333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:47.033577 3333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:47.034938 3333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:44627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:44627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:47.035218 3333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:47.035427 3333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:47.039726 3338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.039759 3333 server_base.cc:1034] running on GCE node
W20260927 11:11:47.039754 3339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.039726 3341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.040103 3333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:47.040364 3333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:47.041504 3333 hybrid_clock.cc:648] HybridClock initialized: now 1790507507041499 us; error 34 us; skew 500 ppm
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:47.043762 3333 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:11:47.044147 3333 webserver.cc:460] Webserver started at http://127.17.95.254:39165/ using document root <none> and password file <none>
I20260927 11:11:47.044360 3333 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:47.044427 3333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:47.044584 3333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:47.045584 3333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "4a601e3251a24ffb82dfcbf13057f98b"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "c1181dd4115bf301c4bd358462e80136"
server_key_iv: "76fa4826b59767962d08600cc2f09644"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.045922 3333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "4a601e3251a24ffb82dfcbf13057f98b"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "c1181dd4115bf301c4bd358462e80136"
server_key_iv: "76fa4826b59767962d08600cc2f09644"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.047220 3333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:11:47.047978 3348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.048148 3333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:47.048235 3333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "4a601e3251a24ffb82dfcbf13057f98b"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "c1181dd4115bf301c4bd358462e80136"
server_key_iv: "76fa4826b59767962d08600cc2f09644"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.048321 3333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:47.056084 3333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:47.058228 3333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:47.058378 3333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:47.063603 3333 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:44627
I20260927 11:11:47.063648 3400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:44627 every 8 connection(s)
I20260927 11:11:47.064023 3333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:47.065022 3401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:47.068087 3401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b: Bootstrap starting.
I20260927 11:11:47.068284 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3333
I20260927 11:11:47.068413 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:47.068650 17791 external_mini_cluster.cc:1428] Setting key eb3237fe3b71d92bee971fae48c22b1c
I20260927 11:11:47.068811 3401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:47.069149 3401 log.cc:824] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:47.069870 3401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b: No bootstrap required, opened a new log
I20260927 11:11:47.072579 3401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a601e3251a24ffb82dfcbf13057f98b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44627 } }
I20260927 11:11:47.072745 3401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:47.072774 3401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a601e3251a24ffb82dfcbf13057f98b, State: Initialized, Role: FOLLOWER
I20260927 11:11:47.072926 3401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a601e3251a24ffb82dfcbf13057f98b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44627 } }
I20260927 11:11:47.073012 3401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:47.073042 3401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:47.073125 3401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:47.073747 3401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a601e3251a24ffb82dfcbf13057f98b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44627 } }
I20260927 11:11:47.073879 3401 leader_election.cc:302] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a601e3251a24ffb82dfcbf13057f98b; no voters:
I20260927 11:11:47.074052 3401 leader_election.cc:288] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:47.074203 3407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:47.074337 3401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:47.074376 3407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [term 1 LEADER]: Becoming Leader. State: Replica: 4a601e3251a24ffb82dfcbf13057f98b, State: Running, Role: LEADER
I20260927 11:11:47.074522 3407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a601e3251a24ffb82dfcbf13057f98b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44627 } }
I20260927 11:11:47.074874 3407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a601e3251a24ffb82dfcbf13057f98b. Latest consensus state: current_term: 1 leader_uuid: "4a601e3251a24ffb82dfcbf13057f98b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a601e3251a24ffb82dfcbf13057f98b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44627 } } }
I20260927 11:11:47.074942 3407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:47.074976 3408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a601e3251a24ffb82dfcbf13057f98b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a601e3251a24ffb82dfcbf13057f98b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 44627 } } }
I20260927 11:11:47.075052 3408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:47.075619 3410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:47.076186 3410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:47.080719 3410 catalog_manager.cc:1347] Generated new cluster ID: 1f09b07bc0f94dd6881cc9138b91a0b9
I20260927 11:11:47.080770 3410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:47.082037 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.069091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.17.95.254:44627)
0927 11:11:47.069277 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.069288 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.069606 (+ 318us) server_negotiation.cc:407] Connection header received
0927 11:11:47.070221 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.070230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.070287 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.070462 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:47.071379 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.072806 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.073903 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.074553 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.074646 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.077957 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:47.077961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:47.077961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:47.077972 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:47.080606 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.080999 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.081002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.081002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.081091 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.081297 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.081298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.081298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.081515 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:47.081689 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.081752 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.081879 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
I20260927 11:11:47.092751 3410 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:47.093237 3410 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:47.101233 3410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b: Generated new TSK 0
I20260927 11:11:47.101400 3410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:11:47.133452 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44627
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:47.218192 3427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:47.218423 3427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:47.218457 3427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:47.219684 3427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:47.219743 3427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:47.219784 3427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:47.221140 3427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:47.221378 3427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:47.221590 3427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:47.227144 3433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.227154 3435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.227208 3427 server_base.cc:1034] running on GCE node
W20260927 11:11:47.227193 3432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.227609 3427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:47.227838 3427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:47.228994 3427 hybrid_clock.cc:648] HybridClock initialized: now 1790507507228974 us; error 30 us; skew 500 ppm
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:47.231356 3427 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:11:47.231756 3427 webserver.cc:460] Webserver started at http://127.17.95.193:41571/ using document root <none> and password file <none>
I20260927 11:11:47.231961 3427 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:47.232024 3427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:47.232170 3427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:47.233108 3427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "fade63f4daae4c24a907bca707957eae"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "54a7b53d8dff2b74d5847502cb0d6d14"
server_key_iv: "3970acb9bb8a3dc867cbc91e484c4ab2"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.233443 3427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "fade63f4daae4c24a907bca707957eae"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "54a7b53d8dff2b74d5847502cb0d6d14"
server_key_iv: "3970acb9bb8a3dc867cbc91e484c4ab2"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.234704 3427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:47.235451 3442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.235617 3427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.235713 3427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "fade63f4daae4c24a907bca707957eae"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "54a7b53d8dff2b74d5847502cb0d6d14"
server_key_iv: "3970acb9bb8a3dc867cbc91e484c4ab2"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.235787 3427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:47.245783 3427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:47.247725 3427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:47.247874 3427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:47.248088 3427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:47.248377 3427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:47.248430 3427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.248471 3427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:47.248498 3427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.254325 3427 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:41861
I20260927 11:11:47.254376 3555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:41861 every 8 connection(s)
I20260927 11:11:47.254663 3427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:47.258128 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3427
I20260927 11:11:47.258253 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:11:47.258467 17791 external_mini_cluster.cc:1428] Setting key 7e8d9f17a7d5015effae5f28e127473e
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:47.263974 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.255301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:53899 (local address 127.17.95.254:44627)
0927 11:11:47.255379 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.255381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.255567 (+ 186us) server_negotiation.cc:407] Connection header received
0927 11:11:47.255994 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.255995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.256005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.256062 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:47.256919 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.257454 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.258478 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.258755 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.258838 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.261202 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:47.261203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:47.261204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:47.261209 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:47.262608 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.263018 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.263019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.263020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.263120 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.263359 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.263359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.263360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.263470 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:47.263538 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.263806 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.263888 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 11:11:47.264161 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.255274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44627 (local address 127.17.95.193:53899)
0927 11:11:47.255452 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.255486 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.255918 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.256124 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.256127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.256662 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:47.256839 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.256844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.257532 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.257533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.258404 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.258406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.258849 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.258850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.258972 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.259501 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:47.259509 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:47.261089 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:47.262726 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:47.262732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:47.262734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:47.262943 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:47.263175 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:47.263176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:47.263176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:47.263286 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:47.263540 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:47.263547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:47.263675 (+ 128us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.263773 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.263915 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260927 11:11:47.264568 3556 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44627
I20260927 11:11:47.264672 3556 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:47.264856 3556 heartbeater.cc:511] Master 127.17.95.254:44627 requested a full tablet report, sending...
I20260927 11:11:47.265383 3365 ts_manager.cc:194] Registered new tserver with Master: fade63f4daae4c24a907bca707957eae (127.17.95.193:41861)
I20260927 11:11:47.265976 3365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:53899
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:11:47.282012 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44627
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:47.361088 3561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:47.361274 3561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:47.361296 3561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:47.362438 3561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:47.362480 3561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:47.362505 3561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:47.363968 3561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:47.364189 3561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:47.364420 3561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:47.369542 3569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.369589 3566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.369542 3567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.369617 3561 server_base.cc:1034] running on GCE node
I20260927 11:11:47.370045 3561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:47.370276 3561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:47.371435 3561 hybrid_clock.cc:648] HybridClock initialized: now 1790507507371423 us; error 24 us; skew 500 ppm
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:47.373670 3561 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:11:47.374073 3561 webserver.cc:460] Webserver started at http://127.17.95.194:46489/ using document root <none> and password file <none>
I20260927 11:11:47.374291 3561 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:47.374351 3561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:47.374487 3561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:47.375686 3561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "b5440df4385848269d34abf9e9bc1016"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "14776bea8f12d06d4a2e37768b0fa9ea"
server_key_iv: "1c705dbc39fcc181e6ad11195acf1b6b"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.376056 3561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "b5440df4385848269d34abf9e9bc1016"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "14776bea8f12d06d4a2e37768b0fa9ea"
server_key_iv: "1c705dbc39fcc181e6ad11195acf1b6b"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.377250 3561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:47.377949 3576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.378142 3561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:47.378242 3561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "b5440df4385848269d34abf9e9bc1016"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "14776bea8f12d06d4a2e37768b0fa9ea"
server_key_iv: "1c705dbc39fcc181e6ad11195acf1b6b"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.378326 3561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:47.388517 3561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:47.390234 3561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:47.390377 3561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:47.390581 3561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:47.390882 3561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:47.390942 3561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.390978 3561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:47.391003 3561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.396524 3561 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:37589
I20260927 11:11:47.396570 3689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:37589 every 8 connection(s)
I20260927 11:11:47.396844 3561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:47.405172 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.397475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:56503 (local address 127.17.95.254:44627)
0927 11:11:47.397576 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.397579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.397740 (+ 161us) server_negotiation.cc:407] Connection header received
0927 11:11:47.398253 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.398255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.398266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.398317 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:47.399324 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.399836 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.400782 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.400963 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.401029 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.402930 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:47.402931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:47.402931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:47.402937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:47.404096 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.404428 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.404428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.404429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.404473 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.404689 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.404689 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.404690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.404780 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:47.404828 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.405056 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.405112 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 11:11:47.405337 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.397435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44627 (local address 127.17.95.194:56503)
0927 11:11:47.397606 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.397645 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.398154 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.398392 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.398395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.398976 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:47.399231 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.399235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.399918 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.399919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.400690 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.400692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.401032 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.401032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.401152 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.401548 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:47.401555 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:47.402852 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:47.404165 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:47.404170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:47.404183 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:47.404369 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:47.404541 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:47.404541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:47.404542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:47.404636 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:47.404832 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:47.404836 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:47.404954 (+ 118us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.405030 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.405149 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260927 11:11:47.405777 3690 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44627
I20260927 11:11:47.405858 3690 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:47.405992 3690 heartbeater.cc:511] Master 127.17.95.254:44627 requested a full tablet report, sending...
I20260927 11:11:47.406373 3365 ts_manager.cc:194] Registered new tserver with Master: b5440df4385848269d34abf9e9bc1016 (127.17.95.194:37589)
I20260927 11:11:47.406435 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3561
I20260927 11:11:47.406548 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:11:47.406728 17791 external_mini_cluster.cc:1428] Setting key 3e5d41c0a538fa4760041d5ca12583c0
I20260927 11:11:47.406827 3365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:56503
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:11:47.430335 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44627
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:47.508693 3695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:47.508908 3695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:47.508944 3695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:47.510051 3695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:47.510113 3695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:47.510154 3695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:47.511552 3695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:47.511801 3695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:47.512013 3695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:47.516582 3700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.516687 3701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.516907 3703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.516932 3695 server_base.cc:1034] running on GCE node
I20260927 11:11:47.517195 3695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:47.517464 3695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:47.518648 3695 hybrid_clock.cc:648] HybridClock initialized: now 1790507507518638 us; error 41 us; skew 500 ppm
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:47.521155 3695 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:11:47.521575 3695 webserver.cc:460] Webserver started at http://127.17.95.195:33377/ using document root <none> and password file <none>
I20260927 11:11:47.521785 3695 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:47.521847 3695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:47.521989 3695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:47.522994 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "56ea3570217c460ebde8e6e70ca46bb8"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "942081535c0b1d14a3de3cc9ab1a0e33"
server_key_iv: "a056c0d8c7575d796bb27638a6ab17dc"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.523411 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "56ea3570217c460ebde8e6e70ca46bb8"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "942081535c0b1d14a3de3cc9ab1a0e33"
server_key_iv: "a056c0d8c7575d796bb27638a6ab17dc"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.524711 3695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:47.525379 3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.525552 3695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:47.525652 3695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "56ea3570217c460ebde8e6e70ca46bb8"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "942081535c0b1d14a3de3cc9ab1a0e33"
server_key_iv: "a056c0d8c7575d796bb27638a6ab17dc"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.525732 3695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:47.538066 3695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:47.540076 3695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:47.540195 3695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:47.540359 3695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:47.540577 3695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:47.540608 3695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.540628 3695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:47.540642 3695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.546242 3695 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35251
I20260927 11:11:47.546320 3823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35251 every 8 connection(s)
I20260927 11:11:47.546626 3695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
Sep 27 11:11:47 dist-test-slave-6j0v krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507507, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:47.554719 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3695
I20260927 11:11:47.554848 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:11:47.555063 17791 external_mini_cluster.cc:1428] Setting key be0aab797621373e89f416e381302419
I20260927 11:11:47.555709 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.547377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:40759 (local address 127.17.95.254:44627)
0927 11:11:47.547442 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.547444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.547676 (+ 232us) server_negotiation.cc:407] Connection header received
0927 11:11:47.548135 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.548136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.548146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.548203 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:47.549108 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.549591 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.550662 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.550837 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.550889 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.552946 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:47.552947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:47.552947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:47.552953 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:47.554173 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.554551 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.554552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.554553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.554630 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.554933 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.554934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.554934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.555119 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:47.555203 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.555526 (+ 323us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.555620 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:11:47.555974 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.547348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:44627 (local address 127.17.95.195:40759)
0927 11:11:47.547526 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.547565 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.548045 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.548263 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.548266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.548826 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:47.549022 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.549026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.549678 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.549679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.550593 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.550595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.550903 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.550903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.550993 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.551505 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:47.551515 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:47.552869 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:47.554247 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:47.554252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:47.554254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:47.554454 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:47.554722 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:47.554722 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:47.554723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:47.554860 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:47.555241 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:47.555248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:47.555377 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.555504 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.555693 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260927 11:11:47.556514 3824 heartbeater.cc:347] Connected to a master server at 127.17.95.254:44627
I20260927 11:11:47.556615 3824 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:47.556758 3824 heartbeater.cc:511] Master 127.17.95.254:44627 requested a full tablet report, sending...
I20260927 11:11:47.557062 3365 ts_manager.cc:194] Registered new tserver with Master: 56ea3570217c460ebde8e6e70ca46bb8 (127.17.95.195:35251)
I20260927 11:11:47.557442 3365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:40759
I20260927 11:11:47.566990 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:47.575125 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.568426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.17.95.254:44627)
0927 11:11:47.568510 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.568512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.568629 (+ 117us) server_negotiation.cc:407] Connection header received
0927 11:11:47.568711 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.568712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.568725 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.568786 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:47.569399 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.569865 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.570603 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.570803 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.570891 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.572317 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:47.572318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:47.572318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:47.572325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:47.573782 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.574223 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.574224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.574224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.574282 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:47.574526 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:47.574527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:47.574528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:47.574676 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:47.574801 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.574936 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.574998 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 11:11:47.576886 3365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:47.578540 3365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:47.584724 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.580591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:37589 (local address 127.0.0.1:40550)
0927 11:11:47.580737 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.580792 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.580863 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.581148 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.581150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.581152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:47.581227 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.581229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.582551 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.582552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.583687 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.583690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.584422 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.584423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.584503 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.584516 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.584604 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.584650 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":55,"threads_started":1}
I20260927 11:11:47.584756 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.580713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.17.95.194:37589)
0927 11:11:47.580950 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.580954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.580964 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:11:47.581004 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.581006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.581022 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.581335 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:47.581415 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.582426 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.583771 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.584334 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.584406 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.584458 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.584617 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.584656 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":120,"threads_started":1}
I20260927 11:11:47.585062 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.580830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41861 (local address 127.0.0.1:38838)
0927 11:11:47.581022 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.581032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.581075 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.581503 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.581504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.581506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:47.581560 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.581561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.583001 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.583002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.583973 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.583975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.584842 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.584844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.584908 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.584912 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.584958 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.584997 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":38,"threads_started":1}
I20260927 11:11:47.585386 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.580943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.17.95.193:41861)
0927 11:11:47.581330 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.581334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.581342 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:11:47.581373 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.581375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.581386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.581653 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:47.581731 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.582893 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.584049 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.584728 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.585171 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.585248 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.585272 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.585309 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":72,"threads_started":1}
I20260927 11:11:47.585891 3624 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf2942ebc744d6990cdd4222d26acd (DEFAULT_TABLE table=test-table [id=d6d4612c6777455e908baeca80405c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:47.585942 3490 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf2942ebc744d6990cdd4222d26acd (DEFAULT_TABLE table=test-table [id=d6d4612c6777455e908baeca80405c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:47.586115 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.580944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35251 (local address 127.0.0.1:52966)
0927 11:11:47.581106 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.581117 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.581168 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.581610 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.581611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.581613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:47.581653 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.581654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.584046 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.584047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.584905 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.584908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.585885 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.585885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.585949 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.585953 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.586000 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.586039 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":9,"thread_start_us":49,"threads_started":1}
I20260927 11:11:47.586275 3490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf2942ebc744d6990cdd4222d26acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:47.586264 3624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf2942ebc744d6990cdd4222d26acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:47.586400 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.581102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.17.95.195:35251)
0927 11:11:47.581291 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.581296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.581324 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:11:47.581364 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.581366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.581381 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.581471 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:47.581738 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.582888 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.584983 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.585694 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.585798 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.585852 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.586247 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.586293 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260927 11:11:47.587678 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf2942ebc744d6990cdd4222d26acd (DEFAULT_TABLE table=test-table [id=d6d4612c6777455e908baeca80405c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:47.588009 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf2942ebc744d6990cdd4222d26acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:47.590020 3838 tablet_bootstrap.cc:492] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Bootstrap starting.
I20260927 11:11:47.590696 3838 tablet_bootstrap.cc:654] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:47.590687 3840 tablet_bootstrap.cc:492] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Bootstrap starting.
I20260927 11:11:47.591252 3839 tablet_bootstrap.cc:492] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Bootstrap starting.
I20260927 11:11:47.591370 3838 log.cc:824] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:47.591425 3840 tablet_bootstrap.cc:654] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:47.591884 3839 tablet_bootstrap.cc:654] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:47.591992 3840 log.cc:824] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:47.592237 3838 tablet_bootstrap.cc:492] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: No bootstrap required, opened a new log
I20260927 11:11:47.592355 3838 ts_tablet_manager.cc:1397] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:47.592458 3839 log.cc:824] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:47.592839 3840 tablet_bootstrap.cc:492] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: No bootstrap required, opened a new log
I20260927 11:11:47.592880 3840 ts_tablet_manager.cc:1397] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:11:47.593240 3839 tablet_bootstrap.cc:492] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: No bootstrap required, opened a new log
I20260927 11:11:47.593292 3839 ts_tablet_manager.cc:1397] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:47.593957 3838 raft_consensus.cc:348] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.594097 3838 raft_consensus.cc:374] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:47.594121 3838 raft_consensus.cc:729] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fade63f4daae4c24a907bca707957eae, State: Initialized, Role: FOLLOWER
I20260927 11:11:47.594242 3838 consensus_queue.cc:260] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.594547 3838 ts_tablet_manager.cc:1428] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:47.594568 3556 heartbeater.cc:503] Master 127.17.95.254:44627 was elected leader, sending a full tablet report...
I20260927 11:11:47.594761 3840 raft_consensus.cc:348] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.594934 3840 raft_consensus.cc:374] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:47.594969 3840 raft_consensus.cc:729] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ea3570217c460ebde8e6e70ca46bb8, State: Initialized, Role: FOLLOWER
I20260927 11:11:47.595100 3840 consensus_queue.cc:260] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.595191 3839 raft_consensus.cc:348] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.595364 3840 ts_tablet_manager.cc:1428] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260927 11:11:47.595356 3839 raft_consensus.cc:374] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:47.595395 3839 raft_consensus.cc:729] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5440df4385848269d34abf9e9bc1016, State: Initialized, Role: FOLLOWER
I20260927 11:11:47.595396 3824 heartbeater.cc:503] Master 127.17.95.254:44627 was elected leader, sending a full tablet report...
I20260927 11:11:47.595534 3839 consensus_queue.cc:260] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.595800 3839 ts_tablet_manager.cc:1428] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:11:47.595832 3690 heartbeater.cc:503] Master 127.17.95.254:44627 was elected leader, sending a full tablet report...
W20260927 11:11:47.647352 3691 tablet.cc:2369] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:11:47.744071 3844 raft_consensus.cc:482] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:47.744230 3844 raft_consensus.cc:504] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.744593 3844 leader_election.cc:288] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5440df4385848269d34abf9e9bc1016 (127.17.95.194:37589), 56ea3570217c460ebde8e6e70ca46bb8 (127.17.95.195:35251)
I20260927 11:11:47.747816 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.744978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35251 (local address 127.17.95.193:55649)
0927 11:11:47.745115 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.745133 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.745182 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.745333 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.745334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.745335 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:47.745410 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.745411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.746104 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.746105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.746913 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.746915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.747554 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.747555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.747628 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.747638 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.747680 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.747711 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260927 11:11:47.747820 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.745076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:55649 (local address 127.17.95.195:35251)
0927 11:11:47.745163 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.745165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.745175 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:11:47.745242 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.745243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.745252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.745320 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:47.745485 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.746019 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.747006 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.747469 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.747556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.747566 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.747704 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.747749 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 11:11:47.747895 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.744855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:33971 (local address 127.17.95.194:37589)
0927 11:11:47.744939 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.744941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.745002 (+ 61us) server_negotiation.cc:407] Connection header received
0927 11:11:47.745032 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.745033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.745045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.745115 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:47.745338 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.745877 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.747029 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.747483 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.747572 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.747580 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.747775 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.747816 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 11:11:47.747949 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.744784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:37589 (local address 127.17.95.193:33971)
0927 11:11:47.744854 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.744885 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:47.744937 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:47.745171 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:47.745172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:47.745175 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:47.745255 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.745257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.745964 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.745965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.746912 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:47.746914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.747592 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.747593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:47.747695 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.747699 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:47.747760 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 11:11:47.747798 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 11:11:47.748395 3644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf2942ebc744d6990cdd4222d26acd" candidate_uuid: "fade63f4daae4c24a907bca707957eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5440df4385848269d34abf9e9bc1016" is_pre_election: true
I20260927 11:11:47.748404 3778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf2942ebc744d6990cdd4222d26acd" candidate_uuid: "fade63f4daae4c24a907bca707957eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56ea3570217c460ebde8e6e70ca46bb8" is_pre_election: true
I20260927 11:11:47.748540 3644 raft_consensus.cc:2454] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fade63f4daae4c24a907bca707957eae in term 0.
I20260927 11:11:47.748556 3778 raft_consensus.cc:2454] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fade63f4daae4c24a907bca707957eae in term 0.
I20260927 11:11:47.748754 3444 leader_election.cc:302] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56ea3570217c460ebde8e6e70ca46bb8, fade63f4daae4c24a907bca707957eae; no voters:
I20260927 11:11:47.748878 3844 raft_consensus.cc:2781] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:47.748982 3844 raft_consensus.cc:482] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:47.749023 3844 raft_consensus.cc:3037] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:47.749774 3844 raft_consensus.cc:504] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.749933 3844 leader_election.cc:288] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [CANDIDATE]: Term 1 election: Requested vote from peers b5440df4385848269d34abf9e9bc1016 (127.17.95.194:37589), 56ea3570217c460ebde8e6e70ca46bb8 (127.17.95.195:35251)
I20260927 11:11:47.750138 3778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf2942ebc744d6990cdd4222d26acd" candidate_uuid: "fade63f4daae4c24a907bca707957eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56ea3570217c460ebde8e6e70ca46bb8"
I20260927 11:11:47.750126 3644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf2942ebc744d6990cdd4222d26acd" candidate_uuid: "fade63f4daae4c24a907bca707957eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5440df4385848269d34abf9e9bc1016"
I20260927 11:11:47.750207 3644 raft_consensus.cc:3037] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:47.750222 3778 raft_consensus.cc:3037] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:47.750957 3644 raft_consensus.cc:2454] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fade63f4daae4c24a907bca707957eae in term 1.
I20260927 11:11:47.751061 3778 raft_consensus.cc:2454] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fade63f4daae4c24a907bca707957eae in term 1.
I20260927 11:11:47.751199 3444 leader_election.cc:302] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5440df4385848269d34abf9e9bc1016, fade63f4daae4c24a907bca707957eae; no voters:
I20260927 11:11:47.751286 3844 raft_consensus.cc:2781] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:47.751433 3844 raft_consensus.cc:686] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 1 LEADER]: Becoming Leader. State: Replica: fade63f4daae4c24a907bca707957eae, State: Running, Role: LEADER
I20260927 11:11:47.751556 3844 consensus_queue.cc:237] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } }
I20260927 11:11:47.752394 3364 catalog_manager.cc:5495] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae reported cstate change: term changed from 0 to 1, leader changed from <none> to fade63f4daae4c24a907bca707957eae (127.17.95.193). New cstate: current_term: 1 leader_uuid: "fade63f4daae4c24a907bca707957eae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fade63f4daae4c24a907bca707957eae" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 41861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 } health_report { overall_health: UNKNOWN } } }
W20260927 11:11:47.755375 3557 tablet.cc:2369] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:11:47.764117 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.762141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.17.95.193:41861)
0927 11:11:47.762200 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.762202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.762208 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:11:47.762234 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.762235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.762248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.762306 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:47.762412 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.762842 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.763545 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.763711 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.763754 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.763868 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:47.763954 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:47.764014 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.764052 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.764060 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 11:11:47.765591 3644 raft_consensus.cc:1261] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 1 FOLLOWER]: Refusing update from remote peer fade63f4daae4c24a907bca707957eae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:47.765619 3778 raft_consensus.cc:1261] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 1 FOLLOWER]: Refusing update from remote peer fade63f4daae4c24a907bca707957eae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:47.765951 3844 consensus_queue.cc:1035] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "56ea3570217c460ebde8e6e70ca46bb8" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:47.766139 3848 consensus_queue.cc:1035] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5440df4385848269d34abf9e9bc1016" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:47.768201 3851 mvcc.cc:205] Tried to move back new op lower bound from 7333918751805620224 to 7333918751751258112. Current Snapshot: MvccSnapshot[applied={T|T < 7333918751805620224}]
I20260927 11:11:47.770166 3852 mvcc.cc:205] Tried to move back new op lower bound from 7333918751805620224 to 7333918751751258112. Current Snapshot: MvccSnapshot[applied={T|T < 7333918751805620224 or (T in {7333918751805620224})}]
I20260927 11:11:47.770160 3365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:47.770256 3365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:47.770923 3365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a601e3251a24ffb82dfcbf13057f98b: Sending DeleteTablet for 3 replicas of tablet 16bf2942ebc744d6990cdd4222d26acd
I20260927 11:11:47.771196 3624 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf2942ebc744d6990cdd4222d26acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:47 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:40550
I20260927 11:11:47.771399 3861 tablet_replica.cc:330] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: stopping tablet replica
I20260927 11:11:47.771425 3490 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf2942ebc744d6990cdd4222d26acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:47 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:38838
I20260927 11:11:47.771512 3861 raft_consensus.cc:2229] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:47.771615 3862 tablet_replica.cc:330] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: stopping tablet replica
I20260927 11:11:47.771641 3861 raft_consensus.cc:2258] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:47.771754 3862 raft_consensus.cc:2229] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:11:47.771747 3758 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf2942ebc744d6990cdd4222d26acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:47 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:52966
I20260927 11:11:47.771842 3862 raft_consensus.cc:2258] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:47.771929 3863 tablet_replica.cc:330] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: stopping tablet replica
I20260927 11:11:47.772010 3490 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf2942ebc744d6990cdd4222d26acd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:47 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:38838
I20260927 11:11:47.772032 3863 raft_consensus.cc:2229] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:47.772122 3863 raft_consensus.cc:2258] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:47.772331 3862 ts_tablet_manager.cc:1905] T 16bf2942ebc744d6990cdd4222d26acd P fade63f4daae4c24a907bca707957eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:47.772365 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3427
I20260927 11:11:47.773020 3863 ts_tablet_manager.cc:1905] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:47.774613 3863 ts_tablet_manager.cc:1918] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:47.774713 3863 log.cc:1196] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/16bf2942ebc744d6990cdd4222d26acd
I20260927 11:11:47.774710 3861 ts_tablet_manager.cc:1905] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:47.774979 3863 ts_tablet_manager.cc:1939] T 16bf2942ebc744d6990cdd4222d26acd P 56ea3570217c460ebde8e6e70ca46bb8: Deleting consensus metadata
I20260927 11:11:47.775363 3350 catalog_manager.cc:4841] TS 56ea3570217c460ebde8e6e70ca46bb8 (127.17.95.195:35251): tablet 16bf2942ebc744d6990cdd4222d26acd (table test-table [id=d6d4612c6777455e908baeca80405c07]) successfully deleted
I20260927 11:11:47.776571 3861 ts_tablet_manager.cc:1918] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:47.776651 3861 log.cc:1196] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/16bf2942ebc744d6990cdd4222d26acd
I20260927 11:11:47.776886 3861 ts_tablet_manager.cc:1939] T 16bf2942ebc744d6990cdd4222d26acd P b5440df4385848269d34abf9e9bc1016: Deleting consensus metadata
I20260927 11:11:47.777235 3350 catalog_manager.cc:4841] TS b5440df4385848269d34abf9e9bc1016 (127.17.95.194:37589): tablet 16bf2942ebc744d6990cdd4222d26acd (table test-table [id=d6d4612c6777455e908baeca80405c07]) successfully deleted
W20260927 11:11:47.778872 3350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:41861 (error 108)
I20260927 11:11:47.779172 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3561
I20260927 11:11:47.779743 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.779254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:41861 (local address 127.0.0.1:38852)
0927 11:11:47.779657 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:47.779696 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385}
W20260927 11:11:47.783541 3350 catalog_manager.cc:4584] TS fade63f4daae4c24a907bca707957eae (127.17.95.193:41861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16bf2942ebc744d6990cdd4222d26acd: Network error: Client connection negotiation failed: client connection to 127.17.95.193:41861: connect: Connection refused (error 111)
I20260927 11:11:47.784123 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3695
I20260927 11:11:47.788980 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3333
2026-09-27T11:11:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1990 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T11:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:47Z Disabled control of system clock
I20260927 11:11:47.815976 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39293
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:39293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:47.891273 3868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:47.891477 3868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:47.891496 3868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:47.892555 3868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:47.892585 3868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:47.892597 3868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:47.892609 3868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:47.892621 3868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:47.893805 3868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:39293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:47.894001 3868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:47.894197 3868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:47.898569 3874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.898578 3868 server_base.cc:1034] running on GCE node
W20260927 11:11:47.898686 3876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:47.898589 3873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:47.899094 3868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:47.899377 3868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:47.900543 3868 hybrid_clock.cc:648] HybridClock initialized: now 1790507507900532 us; error 26 us; skew 500 ppm
I20260927 11:11:47.901798 3868 webserver.cc:460] Webserver started at http://127.17.95.254:39205/ using document root <none> and password file <none>
I20260927 11:11:47.902036 3868 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:47.902102 3868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:47.902240 3868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:47.903264 3868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "efeede06b59c4a818b8b0068d0a61632"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "26afbade62d5e95695661f4876054caa"
server_key_iv: "8da4588aa80d61f451061121b5ffe76d"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.903604 3868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "efeede06b59c4a818b8b0068d0a61632"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "26afbade62d5e95695661f4876054caa"
server_key_iv: "8da4588aa80d61f451061121b5ffe76d"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.904723 3868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:47.905440 3882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:47.905601 3868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:47.905694 3868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "efeede06b59c4a818b8b0068d0a61632"
format_stamp: "Formatted at 2026-09-27 11:11:47 on dist-test-slave-6j0v"
server_key: "26afbade62d5e95695661f4876054caa"
server_key_iv: "8da4588aa80d61f451061121b5ffe76d"
server_key_version: "encryptionkey@0"
I20260927 11:11:47.905772 3868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:47.911821 3868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:47.912052 3868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:47.912184 3868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:47.915760 3868 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:39293
I20260927 11:11:47.915795 3934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:39293 every 8 connection(s)
I20260927 11:11:47.916092 3868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:47.916877 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:47.918819 3935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: Bootstrap starting.
I20260927 11:11:47.919392 3935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:47.919438 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3868
I20260927 11:11:47.919528 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:47.919693 3935 log.cc:824] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:47.919725 17791 external_mini_cluster.cc:1428] Setting key 0c8590f448ffc37cbf4c35625c2f6680
I20260927 11:11:47.920356 3935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: No bootstrap required, opened a new log
I20260927 11:11:47.922456 3935 raft_consensus.cc:348] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efeede06b59c4a818b8b0068d0a61632" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39293 } }
I20260927 11:11:47.922657 3935 raft_consensus.cc:374] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:47.922693 3935 raft_consensus.cc:729] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efeede06b59c4a818b8b0068d0a61632, State: Initialized, Role: FOLLOWER
I20260927 11:11:47.922821 3935 consensus_queue.cc:260] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efeede06b59c4a818b8b0068d0a61632" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39293 } }
I20260927 11:11:47.922888 3935 raft_consensus.cc:388] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:47.922912 3935 raft_consensus.cc:482] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:47.922967 3935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:47.923565 3935 raft_consensus.cc:504] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efeede06b59c4a818b8b0068d0a61632" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39293 } }
I20260927 11:11:47.923648 3935 leader_election.cc:302] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efeede06b59c4a818b8b0068d0a61632; no voters:
I20260927 11:11:47.923820 3935 leader_election.cc:288] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:47.923868 3940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:47.924013 3940 raft_consensus.cc:686] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [term 1 LEADER]: Becoming Leader. State: Replica: efeede06b59c4a818b8b0068d0a61632, State: Running, Role: LEADER
I20260927 11:11:47.924093 3935 sys_catalog.cc:564] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:47.924182 3940 consensus_queue.cc:237] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efeede06b59c4a818b8b0068d0a61632" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39293 } }
I20260927 11:11:47.925765 3941 sys_catalog.cc:455] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efeede06b59c4a818b8b0068d0a61632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efeede06b59c4a818b8b0068d0a61632" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39293 } } }
I20260927 11:11:47.925761 3942 sys_catalog.cc:455] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efeede06b59c4a818b8b0068d0a61632. Latest consensus state: current_term: 1 leader_uuid: "efeede06b59c4a818b8b0068d0a61632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efeede06b59c4a818b8b0068d0a61632" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39293 } } }
I20260927 11:11:47.925868 3941 sys_catalog.cc:458] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:47.925904 3942 sys_catalog.cc:458] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632 [sys.catalog]: This master's current role is: LEADER
W20260927 11:11:47.926273 3955 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:11:47.926385 3955 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:11:47.926445 3956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:47.926607 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:47.920130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.17.95.254:39293)
0927 11:11:47.920323 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:47.920331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:47.920359 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:11:47.920833 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:47.920842 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:47.920912 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:47.921054 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:47.921217 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.923142 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.924117 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:47.924765 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:47.924892 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:47.925812 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:47.925816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:47.925816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:47.926022 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:47.926065 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:47.926082 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:47.926139 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:47.926326 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 11:11:47.926411 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260927 11:11:47.926609 3956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:47.928018 3956 catalog_manager.cc:1347] Generated new cluster ID: 6e9077ed5f2a4dfb95faa0f96332c5dd
I20260927 11:11:47.928067 3956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:47.937980 3956 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:47.938436 3956 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:47.942220 3956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: Generated new TSK 0
I20260927 11:11:47.942365 3956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:47.948376 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39293
--builtin_ntp_servers=127.17.95.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:48.026765 3959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:48.026984 3959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:48.027015 3959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:48.028170 3959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:48.028236 3959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:48.028276 3959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:48.029626 3959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:48.030014 3959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:48.030304 3959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:48.035661 3967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:48.035634 3964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:48.035643 3965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:48.035709 3959 server_base.cc:1034] running on GCE node
I20260927 11:11:48.036095 3959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:48.036355 3959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:48.037561 3959 hybrid_clock.cc:648] HybridClock initialized: now 1790507508037550 us; error 25 us; skew 500 ppm
I20260927 11:11:48.038972 3959 webserver.cc:460] Webserver started at http://127.17.95.193:34597/ using document root <none> and password file <none>
I20260927 11:11:48.039281 3959 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:48.039366 3959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:48.039525 3959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:48.040712 3959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "351aba2b66b44254a2662cc7f0dc9dca"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "34b43b0f7c7f16f92e11b0b9996c425d"
server_key_iv: "b406947f60a48123242c7eb874d88cb4"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.041081 3959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "351aba2b66b44254a2662cc7f0dc9dca"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "34b43b0f7c7f16f92e11b0b9996c425d"
server_key_iv: "b406947f60a48123242c7eb874d88cb4"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.042423 3959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:48.043380 3973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.043592 3959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:48.043680 3959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "351aba2b66b44254a2662cc7f0dc9dca"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "34b43b0f7c7f16f92e11b0b9996c425d"
server_key_iv: "b406947f60a48123242c7eb874d88cb4"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.043744 3959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:48.061517 3959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:48.061750 3959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:48.061848 3959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:48.062021 3959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:48.062288 3959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:48.062317 3959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.062338 3959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:48.062350 3959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.068049 3959 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:45645
I20260927 11:11:48.068148 4086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:45645 every 8 connection(s)
I20260927 11:11:48.068452 3959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:48.073195 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 3959
I20260927 11:11:48.073305 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:11:48.073495 17791 external_mini_cluster.cc:1428] Setting key 1e9e112556553cd3043b9a93b3466877
I20260927 11:11:48.073500 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.069206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:41593 (local address 127.17.95.254:39293)
0927 11:11:48.069273 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.069276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.069538 (+ 262us) server_negotiation.cc:407] Connection header received
0927 11:11:48.069996 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.069997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.070007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.070058 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:48.070684 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.071233 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.072177 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.072352 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.072401 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.073181 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:48.073183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:48.073183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:48.073188 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:48.073205 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:48.073210 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:48.073270 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.073398 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.073425 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:11:48.073828 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.069216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39293 (local address 127.17.95.193:41593)
0927 11:11:48.069408 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.069457 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.069916 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.070117 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.070120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.070129 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:48.070591 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.070617 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.071308 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.071308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.072106 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.072108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.072447 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.072449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.072559 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.073042 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:48.073042 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:48.073068 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:48.073089 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:48.073304 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:48.073311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:48.073315 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.073397 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.073495 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260927 11:11:48.074258 4087 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39293
I20260927 11:11:48.074348 4087 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:48.074513 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39293
--builtin_ntp_servers=127.17.95.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 11:11:48.074540 4087 heartbeater.cc:511] Master 127.17.95.254:39293 requested a full tablet report, sending...
I20260927 11:11:48.075044 3899 ts_manager.cc:194] Registered new tserver with Master: 351aba2b66b44254a2662cc7f0dc9dca (127.17.95.193:45645)
I20260927 11:11:48.075690 3899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.193:41593
W20260927 11:11:48.154186 4090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:48.154364 4090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:48.154384 4090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:48.155571 4090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:48.155614 4090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:48.155639 4090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:48.156927 4090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:48.157130 4090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:48.157418 4090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:48.162345 4098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:48.162480 4090 server_base.cc:1034] running on GCE node
W20260927 11:11:48.162329 4095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:48.162314 4096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:48.162878 4090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:48.163137 4090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:48.164283 4090 hybrid_clock.cc:648] HybridClock initialized: now 1790507508164263 us; error 37 us; skew 500 ppm
I20260927 11:11:48.165493 4090 webserver.cc:460] Webserver started at http://127.17.95.194:39017/ using document root <none> and password file <none>
I20260927 11:11:48.165750 4090 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:48.165818 4090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:48.165964 4090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:48.166940 4090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "e0c300694e344adc958a77c7fe57ca19"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "2f43b4a2cb6c9c3c7ab051fde50c8f9f"
server_key_iv: "49b713633d29755b61f9ff099b8a4db4"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.167295 4090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "e0c300694e344adc958a77c7fe57ca19"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "2f43b4a2cb6c9c3c7ab051fde50c8f9f"
server_key_iv: "49b713633d29755b61f9ff099b8a4db4"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.168545 4090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:11:48.169241 4104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.169414 4090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:11:48.169504 4090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "e0c300694e344adc958a77c7fe57ca19"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "2f43b4a2cb6c9c3c7ab051fde50c8f9f"
server_key_iv: "49b713633d29755b61f9ff099b8a4db4"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.169584 4090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:48.186704 4090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:48.186976 4090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:48.187191 4090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:48.187422 4090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:48.187739 4090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:48.187790 4090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.187825 4090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:48.187852 4090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.193305 4090 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:42701
I20260927 11:11:48.193360 4217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:42701 every 8 connection(s)
I20260927 11:11:48.193638 4090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:11:48.198271 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.194264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:59247 (local address 127.17.95.254:39293)
0927 11:11:48.194302 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.194304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.194513 (+ 209us) server_negotiation.cc:407] Connection header received
0927 11:11:48.195011 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.195013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.195024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.195089 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:48.195736 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.196224 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.197111 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.197292 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.197353 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.198062 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:48.198063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:48.198063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:48.198067 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:48.198079 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:48.198082 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:48.198130 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.198197 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.198216 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:11:48.198493 4220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.194264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39293 (local address 127.17.95.194:59247)
0927 11:11:48.194410 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.194439 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.194908 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.195193 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.195196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.195204 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:48.195665 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.195669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.196299 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.196300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.197032 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.197033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.197365 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.197365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.197461 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.197939 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:48.197940 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:48.197966 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:48.197987 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:48.198133 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:48.198135 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:48.198137 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.198190 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.198258 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":81,"threads_started":1}
I20260927 11:11:48.198994 4218 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39293
I20260927 11:11:48.199106 4218 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:48.199223 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4090
I20260927 11:11:48.199273 4218 heartbeater.cc:511] Master 127.17.95.254:39293 requested a full tablet report, sending...
I20260927 11:11:48.199329 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:11:48.199537 17791 external_mini_cluster.cc:1428] Setting key 05699e88e146b616509a7bd7cf26a5b5
I20260927 11:11:48.199652 3899 ts_manager.cc:194] Registered new tserver with Master: e0c300694e344adc958a77c7fe57ca19 (127.17.95.194:42701)
I20260927 11:11:48.200172 3899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.194:59247
I20260927 11:11:48.200663 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39293
--builtin_ntp_servers=127.17.95.212:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:48.279531 4221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:48.279728 4221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:48.279752 4221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:48.280880 4221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:48.280923 4221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:48.280977 4221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:48.282218 4221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:48.282447 4221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:48.282729 4221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:48.287552 4226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:48.287561 4229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:48.287586 4227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:48.287864 4221 server_base.cc:1034] running on GCE node
I20260927 11:11:48.288048 4221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:48.288244 4221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:48.289379 4221 hybrid_clock.cc:648] HybridClock initialized: now 1790507508289367 us; error 27 us; skew 500 ppm
I20260927 11:11:48.298811 4221 webserver.cc:460] Webserver started at http://127.17.95.195:40409/ using document root <none> and password file <none>
I20260927 11:11:48.299165 4221 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:48.299234 4221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:48.299371 4221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:48.300475 4221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "d3abd3e1a21341dcb1be824e4d43291c"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "b601d288869c0cbdc92ac2597c738c06"
server_key_iv: "041dd734caaf4e95b2d60305891608e3"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.300854 4221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "d3abd3e1a21341dcb1be824e4d43291c"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "b601d288869c0cbdc92ac2597c738c06"
server_key_iv: "041dd734caaf4e95b2d60305891608e3"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.302275 4221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:11:48.303128 4235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.303295 4221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:11:48.303397 4221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "d3abd3e1a21341dcb1be824e4d43291c"
format_stamp: "Formatted at 2026-09-27 11:11:48 on dist-test-slave-6j0v"
server_key: "b601d288869c0cbdc92ac2597c738c06"
server_key_iv: "041dd734caaf4e95b2d60305891608e3"
server_key_version: "encryptionkey@0"
I20260927 11:11:48.303479 4221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:48.314638 4221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:48.314908 4221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:48.315125 4221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:48.315357 4221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:48.315670 4221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:48.315722 4221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.315761 4221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:48.315788 4221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:48.321312 4221 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:45125
I20260927 11:11:48.321358 4348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:45125 every 8 connection(s)
I20260927 11:11:48.321653 4221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:11:48.325348 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4221
I20260927 11:11:48.325462 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:11:48.325752 17791 external_mini_cluster.cc:1428] Setting key 9c2bf8a2acb62697e300e8735659a62c
I20260927 11:11:48.327219 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.322375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:56343 (local address 127.17.95.254:39293)
0927 11:11:48.322436 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.322438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.322624 (+ 186us) server_negotiation.cc:407] Connection header received
0927 11:11:48.323196 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.323202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.323215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.323285 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:48.324040 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.324597 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.325695 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.325995 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.326069 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.326972 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:48.326973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:48.326973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:48.326978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:48.326992 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:48.326995 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:48.327044 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.327138 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.327165 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 11:11:48.327457 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.322331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39293 (local address 127.17.95.195:56343)
0927 11:11:48.322500 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.322537 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.323017 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.323350 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.323353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.323362 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:48.323913 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.323918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.324678 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.324679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.325608 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.325610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.326107 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.326109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.326282 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.326861 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:48.326861 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:48.326886 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:48.326906 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:48.327046 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:48.327048 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:48.327050 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.327145 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.327214 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260927 11:11:48.327859 4349 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39293
I20260927 11:11:48.327927 4349 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:48.328104 4349 heartbeater.cc:511] Master 127.17.95.254:39293 requested a full tablet report, sending...
I20260927 11:11:48.328454 3899 ts_manager.cc:194] Registered new tserver with Master: d3abd3e1a21341dcb1be824e4d43291c (127.17.95.195:45125)
I20260927 11:11:48.328774 3899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.195:56343
I20260927 11:11:48.337488 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:48.341440 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.338819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.17.95.254:39293)
0927 11:11:48.338876 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.338878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.338996 (+ 118us) server_negotiation.cc:407] Connection header received
0927 11:11:48.339031 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.339032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.339039 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.339143 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:48.339291 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.339685 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.340304 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.340453 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.340502 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.341126 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:48.341127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:48.341127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:48.341131 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:48.341141 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:48.341144 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:48.341214 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.341332 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.341344 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:11:48.343016 3899 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:48.344533 3899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:48.350246 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.346592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45645 (local address 127.0.0.1:54592)
0927 11:11:48.346877 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.346887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.346930 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.347260 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.347261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.347263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:48.347303 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.347304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.348545 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.348546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.349344 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.349346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.349990 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.349991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.350059 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.350072 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.350132 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.350177 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":75,"threads_started":1}
I20260927 11:11:48.350262 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.346746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.17.95.193:45645)
0927 11:11:48.346900 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.346904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.346918 (+ 14us) server_negotiation.cc:407] Connection header received
0927 11:11:48.347020 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.347023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.347035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.347174 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:48.347398 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.348469 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.349403 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.349897 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.349976 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.350060 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.350144 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.350180 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260927 11:11:48.350453 4357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.346361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:45125 (local address 127.0.0.1:53972)
0927 11:11:48.346491 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.346529 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.346584 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.347295 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.347296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.347298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:48.347353 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.347355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.348414 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.348415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.349378 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.349381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.350256 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.350257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.350323 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.350327 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.350353 (+ 26us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.350387 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260927 11:11:48.350827 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.346698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.17.95.195:45125)
0927 11:11:48.347103 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.347109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.347118 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:11:48.347153 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.347155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.347169 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.347426 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:48.347477 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.348313 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.349496 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.350142 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.350608 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.350699 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.350723 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.350754 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":70,"threads_started":1}
I20260927 11:11:48.351054 4021 tablet_service.cc:1467] Processing CreateTablet for tablet 802de8775794403d939adad3b93eb8d8 (DEFAULT_TABLE table=test-table [id=b349dfc2d01b430291b207232c465746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:48.351476 4021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 802de8775794403d939adad3b93eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:48.351703 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.346398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:42701 (local address 127.0.0.1:45582)
0927 11:11:48.346710 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.346719 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.346758 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.346977 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.346980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.346983 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:48.347046 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.347047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.349458 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.349459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.350629 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.350632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.351460 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.351461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.351541 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.351544 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.351600 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.351646 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":93,"threads_started":1}
I20260927 11:11:48.351918 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.346505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.17.95.194:42701)
0927 11:11:48.346744 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.346750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.346761 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:11:48.346827 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.346829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.346845 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.346960 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:48.347158 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.348308 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.350729 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.351344 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.351429 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.351494 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.351799 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.351842 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":108,"threads_started":1}
I20260927 11:11:48.353890 4363 tablet_bootstrap.cc:492] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: Bootstrap starting.
I20260927 11:11:48.354393 4283 tablet_service.cc:1467] Processing CreateTablet for tablet 802de8775794403d939adad3b93eb8d8 (DEFAULT_TABLE table=test-table [id=b349dfc2d01b430291b207232c465746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:48.354571 4363 tablet_bootstrap.cc:654] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:48.354718 4283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 802de8775794403d939adad3b93eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:48.354804 4152 tablet_service.cc:1467] Processing CreateTablet for tablet 802de8775794403d939adad3b93eb8d8 (DEFAULT_TABLE table=test-table [id=b349dfc2d01b430291b207232c465746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:48.354904 4363 log.cc:824] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:48.355154 4152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 802de8775794403d939adad3b93eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:48.355568 4363 tablet_bootstrap.cc:492] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: No bootstrap required, opened a new log
I20260927 11:11:48.355609 4363 ts_tablet_manager.cc:1397] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260927 11:11:48.357456 4363 raft_consensus.cc:348] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.357566 4363 raft_consensus.cc:374] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:48.357587 4363 raft_consensus.cc:729] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351aba2b66b44254a2662cc7f0dc9dca, State: Initialized, Role: FOLLOWER
I20260927 11:11:48.357698 4363 consensus_queue.cc:260] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.357867 4365 tablet_bootstrap.cc:492] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Bootstrap starting.
I20260927 11:11:48.357925 4363 ts_tablet_manager.cc:1428] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 11:11:48.358053 4087 heartbeater.cc:503] Master 127.17.95.254:39293 was elected leader, sending a full tablet report...
I20260927 11:11:48.358467 4365 tablet_bootstrap.cc:654] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:48.358788 4365 log.cc:824] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:48.359390 4368 tablet_bootstrap.cc:492] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Bootstrap starting.
I20260927 11:11:48.359540 4365 tablet_bootstrap.cc:492] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: No bootstrap required, opened a new log
I20260927 11:11:48.359606 4365 ts_tablet_manager.cc:1397] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:11:48.360021 4368 tablet_bootstrap.cc:654] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:48.360349 4368 log.cc:824] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:48.361068 4368 tablet_bootstrap.cc:492] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: No bootstrap required, opened a new log
I20260927 11:11:48.361140 4368 ts_tablet_manager.cc:1397] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:11:48.362329 4365 raft_consensus.cc:348] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.362490 4365 raft_consensus.cc:374] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:48.362526 4365 raft_consensus.cc:729] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3abd3e1a21341dcb1be824e4d43291c, State: Initialized, Role: FOLLOWER
I20260927 11:11:48.362635 4365 consensus_queue.cc:260] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.362886 4365 ts_tablet_manager.cc:1428] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 11:11:48.362932 4349 heartbeater.cc:503] Master 127.17.95.254:39293 was elected leader, sending a full tablet report...
I20260927 11:11:48.363566 4368 raft_consensus.cc:348] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.363713 4368 raft_consensus.cc:374] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:48.363749 4368 raft_consensus.cc:729] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c300694e344adc958a77c7fe57ca19, State: Initialized, Role: FOLLOWER
I20260927 11:11:48.363864 4368 consensus_queue.cc:260] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.364089 4368 ts_tablet_manager.cc:1428] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:48.364158 4218 heartbeater.cc:503] Master 127.17.95.254:39293 was elected leader, sending a full tablet report...
W20260927 11:11:48.444176 4219 tablet.cc:2369] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:11:48.569185 4088 tablet.cc:2369] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 11:11:48.572139 4350 tablet.cc:2369] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:11:48.608453 4366 raft_consensus.cc:482] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:48.608560 4366 raft_consensus.cc:504] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.608896 4366 leader_election.cc:288] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3abd3e1a21341dcb1be824e4d43291c (127.17.95.195:45125), e0c300694e344adc958a77c7fe57ca19 (127.17.95.194:42701)
I20260927 11:11:48.611711 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.609194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:40705 (local address 127.17.95.194:42701)
0927 11:11:48.609245 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.609247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.609284 (+ 37us) server_negotiation.cc:407] Connection header received
0927 11:11:48.609362 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.609362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.609372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.609438 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:48.609579 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.610042 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.610981 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.611355 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.611437 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.611445 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.611600 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.611642 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 11:11:48.611744 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.609076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:42701 (local address 127.17.95.193:40705)
0927 11:11:48.609221 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.609230 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.609294 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.609444 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.609445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.609446 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:48.609518 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.609521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.610135 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.610136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.610888 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.610889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.611437 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.611438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.611508 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.611512 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.611594 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.611631 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260927 11:11:48.611738 4361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.609134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:40671 (local address 127.17.95.195:45125)
0927 11:11:48.609178 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.609180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.609277 (+ 97us) server_negotiation.cc:407] Connection header received
0927 11:11:48.609330 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.609331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.609342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.609391 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:48.609547 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.610029 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.610984 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.611394 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.611474 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.611482 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.611644 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.611676 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:11:48.611764 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.609068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:45125 (local address 127.17.95.193:40671)
0927 11:11:48.609191 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.609201 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.609265 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.609407 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:48.609408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:48.609409 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:48.609480 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.609482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.610114 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.610115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.610893 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:48.610895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.611475 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.611475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:48.611555 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.611559 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:48.611628 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:11:48.611665 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":4,"thread_start_us":63,"threads_started":1}
I20260927 11:11:48.612218 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "802de8775794403d939adad3b93eb8d8" candidate_uuid: "351aba2b66b44254a2662cc7f0dc9dca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c300694e344adc958a77c7fe57ca19" is_pre_election: true
I20260927 11:11:48.612210 4303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "802de8775794403d939adad3b93eb8d8" candidate_uuid: "351aba2b66b44254a2662cc7f0dc9dca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3abd3e1a21341dcb1be824e4d43291c" is_pre_election: true
I20260927 11:11:48.612356 4172 raft_consensus.cc:2454] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351aba2b66b44254a2662cc7f0dc9dca in term 0.
I20260927 11:11:48.612356 4303 raft_consensus.cc:2454] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351aba2b66b44254a2662cc7f0dc9dca in term 0.
I20260927 11:11:48.612564 3976 leader_election.cc:302] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351aba2b66b44254a2662cc7f0dc9dca, e0c300694e344adc958a77c7fe57ca19; no voters:
I20260927 11:11:48.612726 4366 raft_consensus.cc:2781] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:48.612825 4366 raft_consensus.cc:482] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:48.612859 4366 raft_consensus.cc:3037] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:48.613541 4366 raft_consensus.cc:504] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.613677 4366 leader_election.cc:288] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [CANDIDATE]: Term 1 election: Requested vote from peers d3abd3e1a21341dcb1be824e4d43291c (127.17.95.195:45125), e0c300694e344adc958a77c7fe57ca19 (127.17.95.194:42701)
I20260927 11:11:48.613826 4172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "802de8775794403d939adad3b93eb8d8" candidate_uuid: "351aba2b66b44254a2662cc7f0dc9dca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c300694e344adc958a77c7fe57ca19"
I20260927 11:11:48.613829 4303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "802de8775794403d939adad3b93eb8d8" candidate_uuid: "351aba2b66b44254a2662cc7f0dc9dca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3abd3e1a21341dcb1be824e4d43291c"
I20260927 11:11:48.613895 4303 raft_consensus.cc:3037] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:48.613926 4172 raft_consensus.cc:3037] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:48.614733 4303 raft_consensus.cc:2454] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351aba2b66b44254a2662cc7f0dc9dca in term 1.
I20260927 11:11:48.614733 4172 raft_consensus.cc:2454] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351aba2b66b44254a2662cc7f0dc9dca in term 1.
I20260927 11:11:48.614898 3974 leader_election.cc:302] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351aba2b66b44254a2662cc7f0dc9dca, d3abd3e1a21341dcb1be824e4d43291c; no voters:
I20260927 11:11:48.615052 4366 raft_consensus.cc:2781] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:48.615223 4366 raft_consensus.cc:686] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [term 1 LEADER]: Becoming Leader. State: Replica: 351aba2b66b44254a2662cc7f0dc9dca, State: Running, Role: LEADER
I20260927 11:11:48.615335 4366 consensus_queue.cc:237] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } }
I20260927 11:11:48.616150 3899 catalog_manager.cc:5495] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca reported cstate change: term changed from 0 to 1, leader changed from <none> to 351aba2b66b44254a2662cc7f0dc9dca (127.17.95.193). New cstate: current_term: 1 leader_uuid: "351aba2b66b44254a2662cc7f0dc9dca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "351aba2b66b44254a2662cc7f0dc9dca" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 45645 } health_report { overall_health: HEALTHY } } }
I20260927 11:11:48.635277 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:48.633123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.17.95.193:45645)
0927 11:11:48.633184 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:48.633186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:48.633193 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:11:48.633243 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:48.633243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:48.633260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:48.633315 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:48.633427 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.633858 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.634544 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:48.634733 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:48.634778 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:48.634918 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:48.635014 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:48.635132 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:48.635219 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 11:11:48.635227 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:11:48.637070 4303 raft_consensus.cc:1261] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 1 FOLLOWER]: Refusing update from remote peer 351aba2b66b44254a2662cc7f0dc9dca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:48.637070 4172 raft_consensus.cc:1261] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 1 FOLLOWER]: Refusing update from remote peer 351aba2b66b44254a2662cc7f0dc9dca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:48.637449 4374 consensus_queue.cc:1035] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3abd3e1a21341dcb1be824e4d43291c" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:48.637606 4366 consensus_queue.cc:1035] T 802de8775794403d939adad3b93eb8d8 P 351aba2b66b44254a2662cc7f0dc9dca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c300694e344adc958a77c7fe57ca19" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:48.640758 4378 mvcc.cc:205] Tried to move back new op lower bound from 7333918755374505984 to 7333918755288920064. Current Snapshot: MvccSnapshot[applied={T|T < 7333918755374505984 or (T in {7333918755374505984})}]
I20260927 11:11:48.642320 3899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:48.642406 3899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:48.643227 3899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efeede06b59c4a818b8b0068d0a61632: Sending DeleteTablet for 3 replicas of tablet 802de8775794403d939adad3b93eb8d8
I20260927 11:11:48.643544 4283 tablet_service.cc:1514] Processing DeleteTablet for tablet 802de8775794403d939adad3b93eb8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:48 UTC) from {username='slave'} at 127.0.0.1:53972
I20260927 11:11:48.643756 4152 tablet_service.cc:1514] Processing DeleteTablet for tablet 802de8775794403d939adad3b93eb8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:48 UTC) from {username='slave'} at 127.0.0.1:45582
I20260927 11:11:48.643963 4021 tablet_service.cc:1514] Processing DeleteTablet for tablet 802de8775794403d939adad3b93eb8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:48 UTC) from {username='slave'} at 127.0.0.1:54592
I20260927 11:11:48.643996 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3959
I20260927 11:11:48.644667 4387 tablet_replica.cc:330] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: stopping tablet replica
I20260927 11:11:48.644773 4387 raft_consensus.cc:2229] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:48.644873 4387 raft_consensus.cc:2258] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:48.645291 4387 ts_tablet_manager.cc:1905] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:48.646605 4387 ts_tablet_manager.cc:1918] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:48.646680 4387 log.cc:1196] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/802de8775794403d939adad3b93eb8d8
I20260927 11:11:48.646926 4387 ts_tablet_manager.cc:1939] T 802de8775794403d939adad3b93eb8d8 P d3abd3e1a21341dcb1be824e4d43291c: Deleting consensus metadata
I20260927 11:11:48.647146 4388 tablet_replica.cc:330] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: stopping tablet replica
I20260927 11:11:48.647233 4388 raft_consensus.cc:2229] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:48.647326 4388 raft_consensus.cc:2258] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:48.647310 3883 catalog_manager.cc:4841] TS d3abd3e1a21341dcb1be824e4d43291c (127.17.95.195:45125): tablet 802de8775794403d939adad3b93eb8d8 (table test-table [id=b349dfc2d01b430291b207232c465746]) successfully deleted
I20260927 11:11:48.647749 4388 ts_tablet_manager.cc:1905] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:48.649730 4388 ts_tablet_manager.cc:1918] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:48.649811 4388 log.cc:1196] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/802de8775794403d939adad3b93eb8d8
I20260927 11:11:48.650095 4388 ts_tablet_manager.cc:1939] T 802de8775794403d939adad3b93eb8d8 P e0c300694e344adc958a77c7fe57ca19: Deleting consensus metadata
W20260927 11:11:48.650481 3885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:45645 (error 108)
I20260927 11:11:48.650715 3885 catalog_manager.cc:4841] TS e0c300694e344adc958a77c7fe57ca19 (127.17.95.194:42701): tablet 802de8775794403d939adad3b93eb8d8 (table test-table [id=b349dfc2d01b430291b207232c465746]) successfully deleted
W20260927 11:11:48.651052 4360 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:11:48.650885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:45645 (local address 127.0.0.1:44850)
0927 11:11:48.650919 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:48.650929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:48.650960 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:48.651021 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:45645: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":19}
W20260927 11:11:48.651225 3885 catalog_manager.cc:4584] TS 351aba2b66b44254a2662cc7f0dc9dca (127.17.95.193:45645): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 802de8775794403d939adad3b93eb8d8: Network error: Client connection negotiation failed: client connection to 127.17.95.193:45645: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:11:48.651307 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4090
I20260927 11:11:48.656703 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4221
I20260927 11:11:48.661226 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 3868
2026-09-27T11:11:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (871 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:48 dist-test-slave-6j0v krb5kdc[4392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:48 dist-test-slave-6j0v krb5kdc[4392](info): set up 2 sockets
Sep 27 11:11:48 dist-test-slave-6j0v krb5kdc[4392](info): commencing operation
krb5kdc: starting...
W20260927 11:11:49.663383 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:49 dist-test-slave-6j0v krb5kdc[4392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:11:49.771951 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39125
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:39125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:49.849552 4405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:49.849779 4405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:49.849817 4405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:49.850960 4405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:49.851032 4405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:49.851054 4405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:49.851073 4405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:49.851090 4405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:49.852594 4405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:39125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:49.852802 4405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:49.852964 4405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:49.857584 4413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:49.857631 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:49.857594 4410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:49.857800 4405 server_base.cc:1034] running on GCE node
I20260927 11:11:49.858103 4405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:49.858408 4405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:49.859539 4405 hybrid_clock.cc:648] HybridClock initialized: now 1790507509859522 us; error 35 us; skew 500 ppm
Sep 27 11:11:49 dist-test-slave-6j0v krb5kdc[4392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507509, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:49.861902 4405 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:11:49.862339 4405 webserver.cc:460] Webserver started at http://127.17.95.254:43089/ using document root <none> and password file <none>
I20260927 11:11:49.862550 4405 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:49.862614 4405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:49.862763 4405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:49.863762 4405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "e99295939a454835852ca44faaf61125"
format_stamp: "Formatted at 2026-09-27 11:11:49 on dist-test-slave-6j0v"
server_key: "1c4c51005c301e32ad582ce40dfc8f7c"
server_key_iv: "c7e65aaaf1e8338d4995b1e2645ce123"
server_key_version: "encryptionkey@0"
I20260927 11:11:49.864081 4405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "e99295939a454835852ca44faaf61125"
format_stamp: "Formatted at 2026-09-27 11:11:49 on dist-test-slave-6j0v"
server_key: "1c4c51005c301e32ad582ce40dfc8f7c"
server_key_iv: "c7e65aaaf1e8338d4995b1e2645ce123"
server_key_version: "encryptionkey@0"
I20260927 11:11:49.865289 4405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:11:49.865991 4420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:49.866179 4405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:11:49.866286 4405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "e99295939a454835852ca44faaf61125"
format_stamp: "Formatted at 2026-09-27 11:11:49 on dist-test-slave-6j0v"
server_key: "1c4c51005c301e32ad582ce40dfc8f7c"
server_key_iv: "c7e65aaaf1e8338d4995b1e2645ce123"
server_key_version: "encryptionkey@0"
I20260927 11:11:49.866355 4405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:49.881834 4405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:49.883576 4405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:49.883742 4405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:49.887917 4405 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:39125
I20260927 11:11:49.887944 4472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:39125 every 8 connection(s)
I20260927 11:11:49.888280 4405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:49.889092 4473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:49.891237 4473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125: Bootstrap starting.
I20260927 11:11:49.891759 4473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:49.891978 4473 log.cc:824] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:49.892561 4473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125: No bootstrap required, opened a new log
I20260927 11:11:49.894011 4473 raft_consensus.cc:348] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99295939a454835852ca44faaf61125" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39125 } }
I20260927 11:11:49.894133 4473 raft_consensus.cc:374] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:49.894155 4473 raft_consensus.cc:729] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e99295939a454835852ca44faaf61125, State: Initialized, Role: FOLLOWER
I20260927 11:11:49.894240 4473 consensus_queue.cc:260] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99295939a454835852ca44faaf61125" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39125 } }
I20260927 11:11:49.894287 4473 raft_consensus.cc:388] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:49.894304 4473 raft_consensus.cc:482] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:49.894371 4473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:49.894876 4473 raft_consensus.cc:504] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99295939a454835852ca44faaf61125" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39125 } }
I20260927 11:11:49.894980 4473 leader_election.cc:302] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e99295939a454835852ca44faaf61125; no voters:
I20260927 11:11:49.895228 4473 leader_election.cc:288] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:49.895275 4476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:49.895417 4476 raft_consensus.cc:686] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [term 1 LEADER]: Becoming Leader. State: Replica: e99295939a454835852ca44faaf61125, State: Running, Role: LEADER
I20260927 11:11:49.895489 4473 sys_catalog.cc:564] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:49.895550 4476 consensus_queue.cc:237] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99295939a454835852ca44faaf61125" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39125 } }
I20260927 11:11:49.896045 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4405
I20260927 11:11:49.896190 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:49.896288 4477 sys_catalog.cc:455] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e99295939a454835852ca44faaf61125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99295939a454835852ca44faaf61125" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39125 } } }
I20260927 11:11:49.896384 4477 sys_catalog.cc:458] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:49.896417 17791 external_mini_cluster.cc:1428] Setting key 36667b2a761a3418877206ce27d6a556
I20260927 11:11:49.896368 4478 sys_catalog.cc:455] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e99295939a454835852ca44faaf61125. Latest consensus state: current_term: 1 leader_uuid: "e99295939a454835852ca44faaf61125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e99295939a454835852ca44faaf61125" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 39125 } } }
I20260927 11:11:49.896446 4478 sys_catalog.cc:458] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:49.896584 4485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:49.897096 4485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:49.898562 4485 catalog_manager.cc:1347] Generated new cluster ID: 580c4154a726473aa504ae889edb178e
I20260927 11:11:49.898608 4485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:11:49 dist-test-slave-6j0v krb5kdc[4392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:49.906540 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:49.896933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37906 (local address 127.17.95.254:39125)
0927 11:11:49.897169 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:49.897177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:49.897208 (+ 31us) server_negotiation.cc:407] Connection header received
0927 11:11:49.897323 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:49.897330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:49.897386 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:49.897517 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:49.898315 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:49.899353 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:49.900083 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:49.900500 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:49.900611 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:49.902824 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:49.902828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:49.902828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:49.902839 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:49.904995 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:49.905401 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:49.905404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:49.905405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:49.905504 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:49.905718 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:49.905718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:49.905719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:49.905953 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:49.906106 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:49.906216 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 11:11:49.906369 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260927 11:11:49.926913 4485 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:49.927686 4485 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:49.939545 4485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125: Generated new TSK 0
I20260927 11:11:49.939729 4485 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.95.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
Entry for principal kudu/127.17.95.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab.
I20260927 11:11:49.967154 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39125
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:50.045250 4499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.045435 4499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.045459 4499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.046504 4499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.046537 4499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.046562 4499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:50.047940 4499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.193.keytab
--principal=kudu/127.17.95.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.048105 4499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.048266 4499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:50.053331 4505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.053341 4504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.053426 4499 server_base.cc:1034] running on GCE node
W20260927 11:11:50.053346 4507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.053802 4499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.054076 4499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.055229 4499 hybrid_clock.cc:648] HybridClock initialized: now 1790507510055228 us; error 37 us; skew 500 ppm
Sep 27 11:11:50 dist-test-slave-6j0v krb5kdc[4392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507510, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:50.057596 4499 init.cc:377] Logged in from keytab as kudu/127.17.95.193@KRBTEST.COM (short username kudu)
I20260927 11:11:50.058005 4499 webserver.cc:460] Webserver started at http://127.17.95.193:36619/ using document root <none> and password file <none>
I20260927 11:11:50.058218 4499 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.058280 4499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.058425 4499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.059482 4499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "f958917eee0c4dff90af2e7114632393"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "ef56589c8a39a7e1cc8e86eb31ea7a49"
server_key_iv: "6195f0fba7879562c766a03e40ea01e7"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.059816 4499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "f958917eee0c4dff90af2e7114632393"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "ef56589c8a39a7e1cc8e86eb31ea7a49"
server_key_iv: "6195f0fba7879562c766a03e40ea01e7"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.061079 4499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:50.061775 4514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.061934 4499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:50.062024 4499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "f958917eee0c4dff90af2e7114632393"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "ef56589c8a39a7e1cc8e86eb31ea7a49"
server_key_iv: "6195f0fba7879562c766a03e40ea01e7"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.062100 4499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:50.072950 4499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:50.075378 4499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:50.075519 4499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:50.075788 4499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:50.076155 4499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:50.076224 4499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.076277 4499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:50.076305 4499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.082753 4499 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:37635
I20260927 11:11:50.082819 4627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:37635 every 8 connection(s)
I20260927 11:11:50.083177 4499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
Sep 27 11:11:50 dist-test-slave-6j0v krb5kdc[4392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507510, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.193@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:50.091379 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.083844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:48355 (local address 127.17.95.254:39125)
0927 11:11:50.083893 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.083895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.084075 (+ 180us) server_negotiation.cc:407] Connection header received
0927 11:11:50.084512 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.084513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.084524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.084572 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.085419 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.085909 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.086827 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.087014 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.087110 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.089088 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.089090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.089090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:50.089096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.090221 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.090564 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.090565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.090565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.090610 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.090832 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.090833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.090833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.090926 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.090975 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.091248 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.091317 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:11:50.091565 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.083828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39125 (local address 127.17.95.193:48355)
0927 11:11:50.083963 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.083994 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.084418 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.084621 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.084624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.085143 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:50.085346 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.085350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.085986 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.085987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.086751 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.086752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.087128 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.087129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.087250 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.087640 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:50.087648 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:50.089004 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:50.090292 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:50.090298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:50.090299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:50.090504 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:50.090665 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:50.090666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:50.090666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:50.090780 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:50.090979 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:50.090983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:50.091131 (+ 148us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.091232 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.091342 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260927 11:11:50.091771 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4499
I20260927 11:11:50.091866 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:11:50.092032 4628 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39125
I20260927 11:11:50.092073 17791 external_mini_cluster.cc:1428] Setting key c57c72b6a0138dcbe6a4acc11bc05063
I20260927 11:11:50.092140 4628 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:50.092298 4628 heartbeater.cc:511] Master 127.17.95.254:39125 requested a full tablet report, sending...
I20260927 11:11:50.092909 4437 ts_manager.cc:194] Registered new tserver with Master: f958917eee0c4dff90af2e7114632393 (127.17.95.193:37635)
I20260927 11:11:50.093532 4437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.193@KRBTEST.COM'} at 127.17.95.193:48355
WARNING: no policy specified for kudu/127.17.95.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
Entry for principal kudu/127.17.95.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab.
I20260927 11:11:50.114856 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39125
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:50.192128 4633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.192310 4633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.192330 4633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.193429 4633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.193468 4633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.193492 4633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:50.194854 4633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.194.keytab
--principal=kudu/127.17.95.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.195053 4633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.195334 4633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:50.200335 4638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.200390 4633 server_base.cc:1034] running on GCE node
W20260927 11:11:50.200346 4639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.200361 4641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.200668 4633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.200858 4633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.201992 4633 hybrid_clock.cc:648] HybridClock initialized: now 1790507510201989 us; error 39 us; skew 500 ppm
Sep 27 11:11:50 dist-test-slave-6j0v krb5kdc[4392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507510, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:50.204339 4633 init.cc:377] Logged in from keytab as kudu/127.17.95.194@KRBTEST.COM (short username kudu)
I20260927 11:11:50.204672 4633 webserver.cc:460] Webserver started at http://127.17.95.194:43639/ using document root <none> and password file <none>
I20260927 11:11:50.204834 4633 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.204870 4633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.204950 4633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.205876 4633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "d7cc2460f4234757a49b726676b3f287"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "6e7322b541d9852f37f29dd2a2254f22"
server_key_iv: "bedd9daf08ba99d4f21c653b92fa7913"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.206153 4633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "d7cc2460f4234757a49b726676b3f287"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "6e7322b541d9852f37f29dd2a2254f22"
server_key_iv: "bedd9daf08ba99d4f21c653b92fa7913"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.207329 4633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:50.208069 4648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.208266 4633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:50.208356 4633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "d7cc2460f4234757a49b726676b3f287"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "6e7322b541d9852f37f29dd2a2254f22"
server_key_iv: "bedd9daf08ba99d4f21c653b92fa7913"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.208454 4633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:50.217698 4633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:50.219353 4633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:50.219486 4633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:50.219713 4633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:50.219977 4633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:50.220048 4633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.220086 4633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:50.220120 4633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.225680 4633 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:38965
I20260927 11:11:50.225790 4761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:38965 every 8 connection(s)
I20260927 11:11:50.226102 4633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:11:50.229889 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4633
I20260927 11:11:50.230001 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:11:50.230175 17791 external_mini_cluster.cc:1428] Setting key 4459089f6bf3af051dd8b7f8880f6508
Sep 27 11:11:50 dist-test-slave-6j0v krb5kdc[4392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507510, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.194@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:50.235209 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.226813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:54599 (local address 127.17.95.254:39125)
0927 11:11:50.226857 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.226859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.227105 (+ 246us) server_negotiation.cc:407] Connection header received
0927 11:11:50.227609 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.227610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.227623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.227669 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.228541 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.229055 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.230040 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.230246 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.230330 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.232401 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.232403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.232403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:50.232410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.233778 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.234188 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.234189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.234190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.234243 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.234550 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.234550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.234551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.234663 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.234738 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.235000 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.235120 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:11:50.235453 4764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.226791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39125 (local address 127.17.95.194:54599)
0927 11:11:50.226969 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.227000 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.227503 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.227731 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.227735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.228273 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:50.228437 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.228441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.229153 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.229154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.229971 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.229974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.230339 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.230340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.230474 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.230988 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:50.230996 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:50.232310 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:50.233896 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:50.233909 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:50.233911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:50.234111 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:50.234317 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:50.234317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:50.234318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:50.234445 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:50.234729 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:50.234735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:50.234875 (+ 140us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.234979 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.235157 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260927 11:11:50.235913 4762 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39125
I20260927 11:11:50.236008 4762 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:50.236169 4762 heartbeater.cc:511] Master 127.17.95.254:39125 requested a full tablet report, sending...
I20260927 11:11:50.236512 4437 ts_manager.cc:194] Registered new tserver with Master: d7cc2460f4234757a49b726676b3f287 (127.17.95.194:38965)
I20260927 11:11:50.236889 4437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.194@KRBTEST.COM'} at 127.17.95.194:54599
WARNING: no policy specified for kudu/127.17.95.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
Entry for principal kudu/127.17.95.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab.
I20260927 11:11:50.253755 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39125
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:50.328728 4767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.328935 4767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.328974 4767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.330116 4767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.330179 4767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.330224 4767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:50.331902 4767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.195.keytab
--principal=kudu/127.17.95.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.332144 4767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.332369 4767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:50.337416 4772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.337498 4773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.337451 4775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.337594 4767 server_base.cc:1034] running on GCE node
I20260927 11:11:50.337764 4767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.337939 4767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.339113 4767 hybrid_clock.cc:648] HybridClock initialized: now 1790507510339057 us; error 25 us; skew 500 ppm
Sep 27 11:11:50 dist-test-slave-6j0v krb5kdc[4392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507510, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:50.341329 4767 init.cc:377] Logged in from keytab as kudu/127.17.95.195@KRBTEST.COM (short username kudu)
I20260927 11:11:50.341771 4767 webserver.cc:460] Webserver started at http://127.17.95.195:34183/ using document root <none> and password file <none>
I20260927 11:11:50.341964 4767 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.342003 4767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.342191 4767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.343245 4767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "c3d0bcfe34304166b35135cee2254da2"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "c536c06ffc91ea1568bd115b2d10fce2"
server_key_iv: "d57cab72971ca789465295582e5a340e"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.343581 4767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "c3d0bcfe34304166b35135cee2254da2"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "c536c06ffc91ea1568bd115b2d10fce2"
server_key_iv: "d57cab72971ca789465295582e5a340e"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.344731 4767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:50.345431 4782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.345594 4767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:50.345688 4767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "c3d0bcfe34304166b35135cee2254da2"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "c536c06ffc91ea1568bd115b2d10fce2"
server_key_iv: "d57cab72971ca789465295582e5a340e"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.345738 4767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:50.364003 4767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:50.366400 4767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:50.366550 4767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:50.366779 4767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:50.367107 4767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:50.367166 4767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.367199 4767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:50.367225 4767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.372816 4767 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:35485
I20260927 11:11:50.372886 4895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:35485 every 8 connection(s)
I20260927 11:11:50.373175 4767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:11:50.377823 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4767
I20260927 11:11:50.377925 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:11:50.378127 17791 external_mini_cluster.cc:1428] Setting key ef1cea45d6bbc03f42973b71073ad6c8
Sep 27 11:11:50 dist-test-slave-6j0v krb5kdc[4392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507510, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.195@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:11:50.381882 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.373830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:35425 (local address 127.17.95.254:39125)
0927 11:11:50.373871 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.373873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.374114 (+ 241us) server_negotiation.cc:407] Connection header received
0927 11:11:50.374592 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.374593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.374605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.374664 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.375603 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.376076 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.377002 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.377176 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.377225 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.379452 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.379453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.379454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:50.379459 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.380801 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.381190 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.381191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.381191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.381236 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.381401 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.381402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.381402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.381491 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.381539 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.381757 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.381820 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:11:50.382073 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.373810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:39125 (local address 127.17.95.195:35425)
0927 11:11:50.373958 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.373996 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.374475 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.374734 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.374738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.375347 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:50.375525 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.375529 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.376150 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.376151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.376933 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.376935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.377246 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.377246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.377344 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.377901 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:11:50.377910 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:50.379376 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:50.380911 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:50.380917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:50.380918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:50.381128 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:50.381277 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:11:50.381278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:11:50.381278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:11:50.381356 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:11:50.381545 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:50.381549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:50.381653 (+ 104us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.381729 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.381835 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260927 11:11:50.382584 4896 heartbeater.cc:347] Connected to a master server at 127.17.95.254:39125
I20260927 11:11:50.382686 4896 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:50.382843 4896 heartbeater.cc:511] Master 127.17.95.254:39125 requested a full tablet report, sending...
I20260927 11:11:50.383220 4437 ts_manager.cc:194] Registered new tserver with Master: c3d0bcfe34304166b35135cee2254da2 (127.17.95.195:35485)
I20260927 11:11:50.383590 4437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.95.195@KRBTEST.COM'} at 127.17.95.195:35425
I20260927 11:11:50.389588 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:50.396230 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.390768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37908 (local address 127.17.95.254:39125)
0927 11:11:50.390820 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.390827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.390937 (+ 110us) server_negotiation.cc:407] Connection header received
0927 11:11:50.390969 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.390969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.390977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.391014 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.391544 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.391987 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.392770 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.392924 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.392967 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.393962 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.393963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.393963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:50.393968 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.395352 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.395605 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.395605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.395606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.395649 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:50.395756 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:50.395756 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:50.395757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:50.395874 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.396000 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.396088 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.396141 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 11:11:50.397792 4437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:50.399240 4437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:50.405061 4904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.401374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:38965 (local address 127.0.0.1:49668)
0927 11:11:50.401703 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.401712 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.401762 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.401972 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.401972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.401974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:50.402013 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.402015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.403187 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.403188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.403935 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.403938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.404778 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.404832 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.404868 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.404939 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.404988 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":40,"threads_started":1}
I20260927 11:11:50.405169 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.401200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:37635 (local address 127.0.0.1:49802)
0927 11:11:50.401338 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.401373 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.401427 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.401717 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.401720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.401722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:50.401788 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.401791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.403021 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.403022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.404117 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.404120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.404884 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.404974 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.404976 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.405026 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.405054 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":8,"thread_start_us":51,"threads_started":1}
I20260927 11:11:50.405335 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.401538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.17.95.194:38965)
0927 11:11:50.401694 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.401698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.401751 (+ 53us) server_negotiation.cc:407] Connection header received
0927 11:11:50.401822 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.401824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.401838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.401903 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:50.402083 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.403110 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404017 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.404687 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404760 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.404812 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.405210 (+ 398us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.405259 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260927 11:11:50.405759 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.401373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.17.95.193:37635)
0927 11:11:50.401534 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.401539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.401548 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:11:50.401584 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.401585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.401622 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.401712 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:50.401853 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.402923 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404185 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.404807 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404865 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.404934 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.405046 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.405092 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":61,"threads_started":1}
I20260927 11:11:50.406137 4696 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 (DEFAULT_TABLE table=test-table [id=9d9a8c769e68466ba7c139227b459bc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:50.406286 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.401200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:35485 (local address 127.0.0.1:58144)
0927 11:11:50.401450 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.401460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.401512 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.402015 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.402016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.402018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:50.402065 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.402066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.402879 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.402880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.404119 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.404120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.405759 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.405760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.406126 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.406132 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.406190 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.406226 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":4,"thread_start_us":48,"threads_started":1}
I20260927 11:11:50.406503 4696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:50.406483 4562 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 (DEFAULT_TABLE table=test-table [id=9d9a8c769e68466ba7c139227b459bc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:50.406723 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.401271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.17.95.195:35485)
0927 11:11:50.401834 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.401838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.401846 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:11:50.401876 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.401877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.401889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.401953 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:50.402131 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.402776 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.404964 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.405673 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.405745 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.405811 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.406580 (+ 769us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.406659 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":69,"threads_started":1}
I20260927 11:11:50.406831 4562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:50.407809 4830 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 (DEFAULT_TABLE table=test-table [id=9d9a8c769e68466ba7c139227b459bc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:50.408115 4830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:50.409651 4910 tablet_bootstrap.cc:492] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: Bootstrap starting.
I20260927 11:11:50.410036 4911 tablet_bootstrap.cc:492] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Bootstrap starting.
I20260927 11:11:50.410358 4910 tablet_bootstrap.cc:654] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:50.410627 4911 tablet_bootstrap.cc:654] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:50.410964 4910 log.cc:824] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:50.411201 4912 tablet_bootstrap.cc:492] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Bootstrap starting.
I20260927 11:11:50.411289 4911 log.cc:824] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:50.411785 4912 tablet_bootstrap.cc:654] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:50.411973 4911 tablet_bootstrap.cc:492] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: No bootstrap required, opened a new log
I20260927 11:11:50.412045 4911 ts_tablet_manager.cc:1397] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:11:50.412267 4912 log.cc:824] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:50.412957 4912 tablet_bootstrap.cc:492] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: No bootstrap required, opened a new log
I20260927 11:11:50.413007 4912 ts_tablet_manager.cc:1397] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:11:50.413607 4911 raft_consensus.cc:348] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.413749 4911 raft_consensus.cc:374] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:50.413792 4911 raft_consensus.cc:729] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7cc2460f4234757a49b726676b3f287, State: Initialized, Role: FOLLOWER
I20260927 11:11:50.413898 4911 consensus_queue.cc:260] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.414162 4762 heartbeater.cc:503] Master 127.17.95.254:39125 was elected leader, sending a full tablet report...
I20260927 11:11:50.414149 4911 ts_tablet_manager.cc:1428] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 11:11:50.414528 4910 tablet_bootstrap.cc:492] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: No bootstrap required, opened a new log
I20260927 11:11:50.414479 4912 raft_consensus.cc:348] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.414590 4910 ts_tablet_manager.cc:1397] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260927 11:11:50.414626 4912 raft_consensus.cc:374] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:50.414669 4912 raft_consensus.cc:729] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d0bcfe34304166b35135cee2254da2, State: Initialized, Role: FOLLOWER
I20260927 11:11:50.414763 4912 consensus_queue.cc:260] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.415014 4896 heartbeater.cc:503] Master 127.17.95.254:39125 was elected leader, sending a full tablet report...
I20260927 11:11:50.415283 4912 ts_tablet_manager.cc:1428] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 11:11:50.416388 4910 raft_consensus.cc:348] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.416540 4910 raft_consensus.cc:374] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:50.416584 4910 raft_consensus.cc:729] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f958917eee0c4dff90af2e7114632393, State: Initialized, Role: FOLLOWER
I20260927 11:11:50.416708 4910 consensus_queue.cc:260] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.416947 4910 ts_tablet_manager.cc:1428] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 11:11:50.416980 4628 heartbeater.cc:503] Master 127.17.95.254:39125 was elected leader, sending a full tablet report...
I20260927 11:11:50.428023 4917 raft_consensus.cc:482] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:50.428117 4917 raft_consensus.cc:504] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.428498 4917 leader_election.cc:288] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f958917eee0c4dff90af2e7114632393 (127.17.95.193:37635), d7cc2460f4234757a49b726676b3f287 (127.17.95.194:38965)
I20260927 11:11:50.431208 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.428670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:34235 (local address 127.17.95.193:37635)
0927 11:11:50.428704 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.428706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.428728 (+ 22us) server_negotiation.cc:407] Connection header received
0927 11:11:50.428791 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.428792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.428802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.428860 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:50.429014 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.429511 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.430472 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.430837 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.430891 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.430898 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.431116 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.431151 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:11:50.431231 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.428710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:44643 (local address 127.17.95.194:38965)
0927 11:11:50.428751 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.428752 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.428897 (+ 145us) server_negotiation.cc:407] Connection header received
0927 11:11:50.428927 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.428928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.428939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.428991 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:50.429134 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.429583 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.430443 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.430854 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.430959 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.430967 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.431138 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.431176 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 11:11:50.431216 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.428617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:37635 (local address 127.17.95.195:34235)
0927 11:11:50.428671 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.428683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.428726 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.428862 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.428863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.428864 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:50.428945 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.428946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.429585 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.429612 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.430355 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.430358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.430926 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.430927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.431000 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.431003 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.431055 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.431136 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":25}
I20260927 11:11:50.431228 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.428673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:38965 (local address 127.17.95.195:44643)
0927 11:11:50.428824 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.428835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.428884 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.429010 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.429011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.429013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:50.429067 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.429068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.429667 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.429668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.430360 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.430362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.430969 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.430974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.431019 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.431022 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.431122 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.431154 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260927 11:11:50.431516 4582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb0a88df2cc4d47ad9ee30a69220cf3" candidate_uuid: "c3d0bcfe34304166b35135cee2254da2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f958917eee0c4dff90af2e7114632393" is_pre_election: true
I20260927 11:11:50.431684 4582 raft_consensus.cc:2454] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3d0bcfe34304166b35135cee2254da2 in term 0.
I20260927 11:11:50.431717 4716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb0a88df2cc4d47ad9ee30a69220cf3" candidate_uuid: "c3d0bcfe34304166b35135cee2254da2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cc2460f4234757a49b726676b3f287" is_pre_election: true
I20260927 11:11:50.431869 4716 raft_consensus.cc:2454] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3d0bcfe34304166b35135cee2254da2 in term 0.
I20260927 11:11:50.431912 4786 leader_election.cc:302] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d0bcfe34304166b35135cee2254da2, f958917eee0c4dff90af2e7114632393; no voters:
I20260927 11:11:50.432019 4917 raft_consensus.cc:2781] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:50.432090 4917 raft_consensus.cc:482] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:50.432106 4917 raft_consensus.cc:3037] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:50.432679 4917 raft_consensus.cc:504] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.432807 4917 leader_election.cc:288] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [CANDIDATE]: Term 1 election: Requested vote from peers f958917eee0c4dff90af2e7114632393 (127.17.95.193:37635), d7cc2460f4234757a49b726676b3f287 (127.17.95.194:38965)
I20260927 11:11:50.432928 4582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb0a88df2cc4d47ad9ee30a69220cf3" candidate_uuid: "c3d0bcfe34304166b35135cee2254da2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f958917eee0c4dff90af2e7114632393"
I20260927 11:11:50.432952 4716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb0a88df2cc4d47ad9ee30a69220cf3" candidate_uuid: "c3d0bcfe34304166b35135cee2254da2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cc2460f4234757a49b726676b3f287"
I20260927 11:11:50.432996 4582 raft_consensus.cc:3037] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:50.433005 4716 raft_consensus.cc:3037] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:50.433771 4582 raft_consensus.cc:2454] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d0bcfe34304166b35135cee2254da2 in term 1.
I20260927 11:11:50.433780 4716 raft_consensus.cc:2454] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d0bcfe34304166b35135cee2254da2 in term 1.
I20260927 11:11:50.433972 4785 leader_election.cc:302] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d0bcfe34304166b35135cee2254da2, d7cc2460f4234757a49b726676b3f287; no voters:
I20260927 11:11:50.434072 4917 raft_consensus.cc:2781] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:50.434186 4917 raft_consensus.cc:686] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 1 LEADER]: Becoming Leader. State: Replica: c3d0bcfe34304166b35135cee2254da2, State: Running, Role: LEADER
I20260927 11:11:50.434268 4917 consensus_queue.cc:237] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } }
I20260927 11:11:50.434983 4437 catalog_manager.cc:5495] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3d0bcfe34304166b35135cee2254da2 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "c3d0bcfe34304166b35135cee2254da2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d0bcfe34304166b35135cee2254da2" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 35485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 } health_report { overall_health: UNKNOWN } } }
I20260927 11:11:50.440274 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.437753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.17.95.195:35485)
0927 11:11:50.437805 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.437807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.437828 (+ 21us) server_negotiation.cc:407] Connection header received
0927 11:11:50.437908 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.437909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.437924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.437984 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:50.438159 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.438599 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.439361 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.439557 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.439687 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.439876 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:50.440005 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:50.440077 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.440120 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.440139 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 11:11:50.441916 4716 raft_consensus.cc:1261] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 1 FOLLOWER]: Refusing update from remote peer c3d0bcfe34304166b35135cee2254da2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:50.441919 4582 raft_consensus.cc:1261] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393 [term 1 FOLLOWER]: Refusing update from remote peer c3d0bcfe34304166b35135cee2254da2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:50.442221 4920 consensus_queue.cc:1035] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7cc2460f4234757a49b726676b3f287" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 38965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:50.442350 4917 consensus_queue.cc:1035] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f958917eee0c4dff90af2e7114632393" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:50.445063 4924 mvcc.cc:205] Tried to move back new op lower bound from 7333918762767724544 to 7333918762739355648. Current Snapshot: MvccSnapshot[applied={T|T < 7333918762767724544}]
I20260927 11:11:50.445641 4925 mvcc.cc:205] Tried to move back new op lower bound from 7333918762767724544 to 7333918762739355648. Current Snapshot: MvccSnapshot[applied={T|T < 7333918762767724544}]
I20260927 11:11:50.446821 4437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:50.446918 4437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:50.447750 4437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e99295939a454835852ca44faaf61125: Sending DeleteTablet for 3 replicas of tablet 3bb0a88df2cc4d47ad9ee30a69220cf3
I20260927 11:11:50.448110 4562 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:50 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49802
I20260927 11:11:50.448166 4696 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:50 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:49668
I20260927 11:11:50.448230 4830 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:50 UTC) from {username='kudu', principal='kudu/127.17.95.254@KRBTEST.COM'} at 127.0.0.1:58144
I20260927 11:11:50.448352 4933 tablet_replica.cc:330] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: stopping tablet replica
I20260927 11:11:50.448450 4933 raft_consensus.cc:2229] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:50.448557 4933 raft_consensus.cc:2258] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:50.448735 4932 tablet_replica.cc:330] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P f958917eee0c4dff90af2e7114632393: stopping tablet replica
I20260927 11:11:50.448764 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4499
I20260927 11:11:50.449170 4933 ts_tablet_manager.cc:1905] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:50.449611 4934 tablet_replica.cc:330] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: stopping tablet replica
I20260927 11:11:50.449712 4934 raft_consensus.cc:2229] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:11:50.449822 4934 raft_consensus.cc:2258] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:50.450255 4934 ts_tablet_manager.cc:1905] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:50.451157 4933 ts_tablet_manager.cc:1918] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:50.451292 4933 log.cc:1196] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/3bb0a88df2cc4d47ad9ee30a69220cf3
I20260927 11:11:50.451650 4933 ts_tablet_manager.cc:1939] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P d7cc2460f4234757a49b726676b3f287: Deleting consensus metadata
I20260927 11:11:50.452031 4934 ts_tablet_manager.cc:1918] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:50.452105 4934 log.cc:1196] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/3bb0a88df2cc4d47ad9ee30a69220cf3
I20260927 11:11:50.452227 4423 catalog_manager.cc:4841] TS d7cc2460f4234757a49b726676b3f287 (127.17.95.194:38965): tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 (table test-table [id=9d9a8c769e68466ba7c139227b459bc8]) successfully deleted
I20260927 11:11:50.452404 4934 ts_tablet_manager.cc:1939] T 3bb0a88df2cc4d47ad9ee30a69220cf3 P c3d0bcfe34304166b35135cee2254da2: Deleting consensus metadata
W20260927 11:11:50.454957 4424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:37635 (error 108)
I20260927 11:11:50.457856 4423 catalog_manager.cc:4841] TS c3d0bcfe34304166b35135cee2254da2 (127.17.95.195:35485): tablet 3bb0a88df2cc4d47ad9ee30a69220cf3 (table test-table [id=9d9a8c769e68466ba7c139227b459bc8]) successfully deleted
W20260927 11:11:50.458556 4907 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:11:50.458113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:37635 (local address 127.0.0.1:49804)
0927 11:11:50.458168 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.458180 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.458225 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.458524 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:37635: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 11:11:50.458742 4424 catalog_manager.cc:4584] TS f958917eee0c4dff90af2e7114632393 (127.17.95.193:37635): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bb0a88df2cc4d47ad9ee30a69220cf3: Network error: Client connection negotiation failed: client connection to 127.17.95.193:37635: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 11:11:50.458774 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4633
I20260927 11:11:50.465591 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4767
I20260927 11:11:50.470099 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4405
2026-09-27T11:11:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1809 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T11:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:50Z Disabled control of system clock
I20260927 11:11:50.496572 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:32971
--webserver_interface=127.17.95.254
--webserver_port=0
--builtin_ntp_servers=127.17.95.212:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:32971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:50.575721 4939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.575922 4939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.575941 4939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.577020 4939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:50.577052 4939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.577065 4939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.577077 4939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:50.577090 4939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:50.578287 4939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:32971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:32971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.578456 4939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.578668 4939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:50.582868 4947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.582950 4939 server_base.cc:1034] running on GCE node
W20260927 11:11:50.582847 4944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.582899 4945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.583302 4939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.583518 4939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.584694 4939 hybrid_clock.cc:648] HybridClock initialized: now 1790507510584673 us; error 36 us; skew 500 ppm
I20260927 11:11:50.585916 4939 webserver.cc:460] Webserver started at http://127.17.95.254:37141/ using document root <none> and password file <none>
I20260927 11:11:50.586161 4939 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.586232 4939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.586400 4939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.587425 4939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "dc3abf0af4b8445383e25d77fa1561d0"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "262a869039337099f40fbd5cc5ac43b4"
server_key_iv: "3b7efb542ceb04b4fb50e1a96b68ceb2"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.587769 4939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "dc3abf0af4b8445383e25d77fa1561d0"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "262a869039337099f40fbd5cc5ac43b4"
server_key_iv: "3b7efb542ceb04b4fb50e1a96b68ceb2"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.588950 4939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:50.589633 4953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.589804 4939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:50.589896 4939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "dc3abf0af4b8445383e25d77fa1561d0"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "262a869039337099f40fbd5cc5ac43b4"
server_key_iv: "3b7efb542ceb04b4fb50e1a96b68ceb2"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.589975 4939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:50.600476 4939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:50.600721 4939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:50.600852 4939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:50.604269 4939 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:32971
I20260927 11:11:50.604295 5005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:32971 every 8 connection(s)
I20260927 11:11:50.604573 4939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:50.605531 5006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:50.607661 5006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0: Bootstrap starting.
I20260927 11:11:50.608254 5006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:50.608531 5006 log.cc:824] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:50.609189 5006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0: No bootstrap required, opened a new log
I20260927 11:11:50.610647 5006 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc3abf0af4b8445383e25d77fa1561d0" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 32971 } }
I20260927 11:11:50.610769 5006 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:50.610791 5006 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3abf0af4b8445383e25d77fa1561d0, State: Initialized, Role: FOLLOWER
I20260927 11:11:50.610931 5006 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc3abf0af4b8445383e25d77fa1561d0" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 32971 } }
I20260927 11:11:50.611006 5006 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:50.611025 5006 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:50.611068 5006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:50.611567 5006 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc3abf0af4b8445383e25d77fa1561d0" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 32971 } }
I20260927 11:11:50.611670 5006 leader_election.cc:302] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc3abf0af4b8445383e25d77fa1561d0; no voters:
I20260927 11:11:50.611836 5006 leader_election.cc:288] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:50.611913 5009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:50.612080 5009 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [term 1 LEADER]: Becoming Leader. State: Replica: dc3abf0af4b8445383e25d77fa1561d0, State: Running, Role: LEADER
I20260927 11:11:50.612104 5006 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:50.612210 5009 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc3abf0af4b8445383e25d77fa1561d0" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 32971 } }
I20260927 11:11:50.612565 5010 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc3abf0af4b8445383e25d77fa1561d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc3abf0af4b8445383e25d77fa1561d0" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 32971 } } }
I20260927 11:11:50.612658 5010 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:50.612665 5011 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc3abf0af4b8445383e25d77fa1561d0. Latest consensus state: current_term: 1 leader_uuid: "dc3abf0af4b8445383e25d77fa1561d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc3abf0af4b8445383e25d77fa1561d0" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 32971 } } }
I20260927 11:11:50.612751 5011 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:50.612879 5018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:50.613559 5018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:50.614208 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 4939
I20260927 11:11:50.614310 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:50.614500 17791 external_mini_cluster.cc:1428] Setting key 0c00acba13195ab3de259776ef86699e
I20260927 11:11:50.615025 5018 catalog_manager.cc:1347] Generated new cluster ID: 8a56369235e94a22bd0ca9628090a6f1
I20260927 11:11:50.615105 5018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:50.620537 5018 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:50.621170 5018 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:50.623891 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.615051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38830 (local address 127.17.95.254:32971)
0927 11:11:50.619361 (+ 4310us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.619367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.619388 (+ 21us) server_negotiation.cc:407] Connection header received
0927 11:11:50.619465 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.619470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.619520 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.619664 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.619817 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.621190 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.621967 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.622404 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.622526 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.623221 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.623224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.623225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:50.623384 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.623416 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:50.623426 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.623500 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.623572 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.623742 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4259,"thread_start_us":98,"threads_started":1}
I20260927 11:11:50.624501 5018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0: Generated new TSK 0
I20260927 11:11:50.624642 5018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:50.630856 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.193:0
--local_ip_for_outbound_sockets=127.17.95.193
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:32971
--builtin_ntp_servers=127.17.95.212:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:50.719110 5030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.719326 5030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.719357 5030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.720443 5030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.720504 5030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.720546 5030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.193
I20260927 11:11:50.721863 5030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.95.193
--webserver_port=0
--tserver_master_addrs=127.17.95.254:32971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.193
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.722098 5030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.722363 5030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:50.728189 5035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.728225 5036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.728479 5038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.728626 5030 server_base.cc:1034] running on GCE node
I20260927 11:11:50.728827 5030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.729079 5030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.730240 5030 hybrid_clock.cc:648] HybridClock initialized: now 1790507510730220 us; error 37 us; skew 500 ppm
I20260927 11:11:50.731588 5030 webserver.cc:460] Webserver started at http://127.17.95.193:43683/ using document root <none> and password file <none>
I20260927 11:11:50.731856 5030 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.731900 5030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.732028 5030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.733153 5030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data/instance:
uuid: "23b373dde4b544ed83b5913cbb728e92"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "02ff8dd73c72da04fc7b5040827c1e4a"
server_key_iv: "61cebdf4cd178e871ca19d562af540d5"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.733525 5030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance:
uuid: "23b373dde4b544ed83b5913cbb728e92"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "02ff8dd73c72da04fc7b5040827c1e4a"
server_key_iv: "61cebdf4cd178e871ca19d562af540d5"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.734777 5030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:11:50.735677 5044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.735827 5030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.735914 5030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal
uuid: "23b373dde4b544ed83b5913cbb728e92"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "02ff8dd73c72da04fc7b5040827c1e4a"
server_key_iv: "61cebdf4cd178e871ca19d562af540d5"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.735966 5030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:50.753573 5030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:50.753862 5030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:50.754010 5030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:50.754225 5030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:50.754526 5030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:50.754578 5030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.754628 5030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:50.754657 5030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.760334 5030 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.193:44777
I20260927 11:11:50.760380 5157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.193:44777 every 8 connection(s)
I20260927 11:11:50.760689 5030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/data/info.pb
I20260927 11:11:50.765686 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.761371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.193:37321 (local address 127.17.95.254:32971)
0927 11:11:50.761433 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.761436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.761662 (+ 226us) server_negotiation.cc:407] Connection header received
0927 11:11:50.762115 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.762115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.762125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.762181 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.762770 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.763318 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.764209 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.764372 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.764419 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.765413 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.765414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.765414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:50.765418 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.765431 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:50.765435 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.765480 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.765598 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.765628 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 11:11:50.765940 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.761336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:32971 (local address 127.17.95.193:37321)
0927 11:11:50.761521 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.761560 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.762035 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.762255 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.762258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.762265 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:50.762694 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.762698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.763403 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.763404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.764130 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.764132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.764629 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.764630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.764748 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.765271 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:50.765272 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:50.765304 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:50.765327 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:50.765511 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:50.765517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:50.765519 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.765588 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.765687 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260927 11:11:50.766479 5158 heartbeater.cc:347] Connected to a master server at 127.17.95.254:32971
I20260927 11:11:50.766562 5158 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:50.766791 5158 heartbeater.cc:511] Master 127.17.95.254:32971 requested a full tablet report, sending...
I20260927 11:11:50.767305 4970 ts_manager.cc:194] Registered new tserver with Master: 23b373dde4b544ed83b5913cbb728e92 (127.17.95.193:44777)
I20260927 11:11:50.768057 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5030
I20260927 11:11:50.768065 4970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.193:37321
I20260927 11:11:50.768162 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-0/wal/instance
I20260927 11:11:50.768381 17791 external_mini_cluster.cc:1428] Setting key 28d5a7fd1658f02ed6517a6aa8563460
I20260927 11:11:50.769266 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.194:0
--local_ip_for_outbound_sockets=127.17.95.194
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:32971
--builtin_ntp_servers=127.17.95.212:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:50.846890 5161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.847087 5161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.847150 5161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.848246 5161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.848306 5161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.848346 5161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.194
I20260927 11:11:50.849689 5161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.95.194
--webserver_port=0
--tserver_master_addrs=127.17.95.254:32971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.194
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.849948 5161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.850255 5161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:11:50.855026 5161 server_base.cc:1034] running on GCE node
W20260927 11:11:50.854995 5169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.854990 5167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.854995 5166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.855487 5161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.855749 5161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.856920 5161 hybrid_clock.cc:648] HybridClock initialized: now 1790507510856892 us; error 41 us; skew 500 ppm
I20260927 11:11:50.858191 5161 webserver.cc:460] Webserver started at http://127.17.95.194:43603/ using document root <none> and password file <none>
I20260927 11:11:50.858400 5161 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.858464 5161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.858600 5161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.859620 5161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data/instance:
uuid: "0b31505fb4d94cf8be938b71b2159100"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "95a51178108a087210ec4b4b81531099"
server_key_iv: "f2522757ca9aef9630bffb1e20ffaa70"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.859939 5161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance:
uuid: "0b31505fb4d94cf8be938b71b2159100"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "95a51178108a087210ec4b4b81531099"
server_key_iv: "f2522757ca9aef9630bffb1e20ffaa70"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.861100 5161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:50.861761 5175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.861907 5161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:50.861996 5161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal
uuid: "0b31505fb4d94cf8be938b71b2159100"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "95a51178108a087210ec4b4b81531099"
server_key_iv: "f2522757ca9aef9630bffb1e20ffaa70"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.862067 5161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:50.878976 5161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:50.879266 5161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:50.879410 5161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:50.879606 5161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:50.879930 5161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:50.879982 5161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.880038 5161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:50.880066 5161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.885413 5161 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.194:34481
I20260927 11:11:50.885494 5288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.194:34481 every 8 connection(s)
I20260927 11:11:50.885807 5161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/data/info.pb
I20260927 11:11:50.891376 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.886528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.194:41283 (local address 127.17.95.254:32971)
0927 11:11:50.886618 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:50.886620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:50.886881 (+ 261us) server_negotiation.cc:407] Connection header received
0927 11:11:50.887482 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:50.887483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:50.887498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:50.887569 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:50.888261 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.888819 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.889864 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.890092 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.890185 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.891001 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:50.891003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:50.891003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:50.891009 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:50.891023 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:50.891028 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:50.891150 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:50.891259 (+ 109us) server_negotiation.cc:299] Negotiation successful
0927 11:11:50.891294 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 11:11:50.891601 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:50.886500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:32971 (local address 127.17.95.194:41283)
0927 11:11:50.886752 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:50.886785 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:50.887313 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:50.887656 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:50.887659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:50.887668 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:50.888155 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.888160 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.888937 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.888940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.889750 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:50.889752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:50.890206 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:50.890207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:50.890336 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:50.890849 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:50.890850 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:50.890876 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:50.890898 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:50.891153 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:50.891160 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:50.891163 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:50.891258 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 11:11:50.891356 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260927 11:11:50.892131 5289 heartbeater.cc:347] Connected to a master server at 127.17.95.254:32971
I20260927 11:11:50.892212 5289 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:50.892472 5289 heartbeater.cc:511] Master 127.17.95.254:32971 requested a full tablet report, sending...
I20260927 11:11:50.892948 4970 ts_manager.cc:194] Registered new tserver with Master: 0b31505fb4d94cf8be938b71b2159100 (127.17.95.194:34481)
I20260927 11:11:50.893325 4970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.194:41283
I20260927 11:11:50.893896 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5161
I20260927 11:11:50.893998 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal/instance
I20260927 11:11:50.894200 17791 external_mini_cluster.cc:1428] Setting key bf8f3b523aa022583ac66161ab793ab3
I20260927 11:11:50.895049 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.95.195:0
--local_ip_for_outbound_sockets=127.17.95.195
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:32971
--builtin_ntp_servers=127.17.95.212:34423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 11:11:50.976094 5292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:50.976294 5292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:50.976317 5292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:50.977458 5292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:50.977501 5292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:50.977527 5292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.95.195
I20260927 11:11:50.978837 5292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.95.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.95.195
--webserver_port=0
--tserver_master_addrs=127.17.95.254:32971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.95.195
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:50.979077 5292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:50.979420 5292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:50.984321 5298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.984329 5300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:50.984589 5297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:50.984702 5292 server_base.cc:1034] running on GCE node
I20260927 11:11:50.984925 5292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:50.985162 5292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:50.986342 5292 hybrid_clock.cc:648] HybridClock initialized: now 1790507510986329 us; error 35 us; skew 500 ppm
I20260927 11:11:50.987722 5292 webserver.cc:460] Webserver started at http://127.17.95.195:45537/ using document root <none> and password file <none>
I20260927 11:11:50.987957 5292 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:50.988025 5292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:50.988179 5292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:50.989262 5292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data/instance:
uuid: "6cf687fb099e411a883ad3a67735a111"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "0efe0a4482220cabe75c83377e1b9a36"
server_key_iv: "fb11dfe912293336f9983efab8f3dc3a"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.989602 5292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance:
uuid: "6cf687fb099e411a883ad3a67735a111"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "0efe0a4482220cabe75c83377e1b9a36"
server_key_iv: "fb11dfe912293336f9983efab8f3dc3a"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.990875 5292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:11:50.991684 5306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:50.991883 5292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 11:11:50.991987 5292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal
uuid: "6cf687fb099e411a883ad3a67735a111"
format_stamp: "Formatted at 2026-09-27 11:11:50 on dist-test-slave-6j0v"
server_key: "0efe0a4482220cabe75c83377e1b9a36"
server_key_iv: "fb11dfe912293336f9983efab8f3dc3a"
server_key_version: "encryptionkey@0"
I20260927 11:11:50.992081 5292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:51.008656 5292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:51.008926 5292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:51.009065 5292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:51.009275 5292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 11:11:51.009586 5292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 11:11:51.009647 5292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:51.009681 5292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 11:11:51.009711 5292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:51.016284 5292 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.195:44347
I20260927 11:11:51.016360 5419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.195:44347 every 8 connection(s)
I20260927 11:11:51.016665 5292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/data/info.pb
I20260927 11:11:51.019645 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5292
I20260927 11:11:51.019757 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal/instance
I20260927 11:11:51.019960 17791 external_mini_cluster.cc:1428] Setting key 24d4206ea8082681cd76a91d5431b01c
I20260927 11:11:51.022267 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.017440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:47329 (local address 127.17.95.254:32971)
0927 11:11:51.017514 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.017516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.017747 (+ 231us) server_negotiation.cc:407] Connection header received
0927 11:11:51.018248 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.018249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.018266 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.018326 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:51.018971 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.019598 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.020949 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.021164 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.021213 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.022074 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:51.022075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:51.022075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:51.022078 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:51.022088 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:51.022091 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:51.022142 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.022195 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.022215 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:11:51.022476 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.017419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:32971 (local address 127.17.95.195:47329)
0927 11:11:51.017615 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.017662 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:51.018145 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:51.018394 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:51.018398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:51.018407 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:11:51.018878 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.018882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.019710 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.019711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.020850 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.020854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.021235 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.021236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.021339 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.021950 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 11:11:51.021950 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:11:51.021978 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 11:11:51.021999 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:11:51.022132 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:11:51.022146 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:11:51.022148 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 11:11:51.022198 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 11:11:51.022268 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260927 11:11:51.022933 5420 heartbeater.cc:347] Connected to a master server at 127.17.95.254:32971
I20260927 11:11:51.023039 5420 heartbeater.cc:464] Registering TS with master...
I20260927 11:11:51.023248 5420 heartbeater.cc:511] Master 127.17.95.254:32971 requested a full tablet report, sending...
I20260927 11:11:51.023655 4970 ts_manager.cc:194] Registered new tserver with Master: 6cf687fb099e411a883ad3a67735a111 (127.17.95.195:44347)
I20260927 11:11:51.024009 4970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.95.195:47329
I20260927 11:11:51.031714 17791 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 11:11:51.036038 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.033101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.17.95.254:32971)
0927 11:11:51.033167 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.033169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.033270 (+ 101us) server_negotiation.cc:407] Connection header received
0927 11:11:51.033333 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.033334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.033349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.033403 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:51.033569 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.034026 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.034767 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.034961 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.035039 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.035708 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:51.035709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:51.035710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:51.035714 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:51.035726 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:51.035730 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:51.035773 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.035916 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.035946 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:11:51.037492 4970 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 11:11:51.039288 4970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 11:11:51.045464 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.041385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.17.95.195:44347)
0927 11:11:51.041544 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.041548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.041745 (+ 197us) server_negotiation.cc:407] Connection header received
0927 11:11:51.042038 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.042040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.042054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.042106 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:51.042332 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.043383 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.044374 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.044974 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.045082 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.045176 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.045323 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.045368 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260927 11:11:51.045748 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.041313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.195:44347 (local address 127.0.0.1:46968)
0927 11:11:51.041631 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.041679 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:51.041954 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:51.042200 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:51.042201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:51.042203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:51.042258 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.042261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.043461 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.043462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.044288 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.044290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.045083 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.045084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.045164 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.045177 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 11:11:51.045651 (+ 474us) client_negotiation.cc:240] Negotiation successful
0927 11:11:51.045700 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260927 11:11:51.045914 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.041314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34481 (local address 127.0.0.1:34096)
0927 11:11:51.041645 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.041677 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:51.041747 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:51.041993 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:51.041995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:51.041997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:51.042055 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.042056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.043361 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.043361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.044563 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.044566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.045692 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.045693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.045754 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.045757 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:51.045807 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 11:11:51.045843 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":50,"thread_start_us":90,"threads_started":1}
I20260927 11:11:51.046219 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.041406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.17.95.194:34481)
0927 11:11:51.041567 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.041573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.041745 (+ 172us) server_negotiation.cc:407] Connection header received
0927 11:11:51.041847 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.041849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.041864 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.041990 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:51.042119 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.043282 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.044658 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.045412 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.045472 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.045658 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.046077 (+ 419us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.046123 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260927 11:11:51.046629 5429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.041522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44777 (local address 127.0.0.1:51650)
0927 11:11:51.041798 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.041807 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:51.041867 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:51.042201 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:51.042202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:51.042203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:51.042258 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.042260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.044273 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.044274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.045283 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.045286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.046193 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.046194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.046254 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.046258 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:51.046520 (+ 262us) client_negotiation.cc:240] Negotiation successful
0927 11:11:51.046565 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":10,"thread_start_us":63,"threads_started":1}
I20260927 11:11:51.046527 5354 tablet_service.cc:1467] Processing CreateTablet for tablet 571390c3d446408a8e9e553ec2419918 (DEFAULT_TABLE table=test-table [id=5ac5a5bcc1a044a58d6b8ad6ffc1a001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:51.046841 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.041734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.17.95.193:44777)
0927 11:11:51.041975 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.041979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.041988 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:11:51.042026 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.042028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.042043 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.042117 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:51.043169 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.044176 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.045386 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.046127 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.046192 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.046276 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.046719 (+ 443us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.046763 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260927 11:11:51.046886 5354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 571390c3d446408a8e9e553ec2419918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:51.046860 5223 tablet_service.cc:1467] Processing CreateTablet for tablet 571390c3d446408a8e9e553ec2419918 (DEFAULT_TABLE table=test-table [id=5ac5a5bcc1a044a58d6b8ad6ffc1a001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:51.047349 5223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 571390c3d446408a8e9e553ec2419918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:51.047554 5092 tablet_service.cc:1467] Processing CreateTablet for tablet 571390c3d446408a8e9e553ec2419918 (DEFAULT_TABLE table=test-table [id=5ac5a5bcc1a044a58d6b8ad6ffc1a001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 11:11:51.047873 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 571390c3d446408a8e9e553ec2419918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:51.049881 5434 tablet_bootstrap.cc:492] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Bootstrap starting.
I20260927 11:11:51.050472 5435 tablet_bootstrap.cc:492] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Bootstrap starting.
I20260927 11:11:51.050522 5434 tablet_bootstrap.cc:654] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:51.050851 5434 log.cc:824] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:51.051136 5435 tablet_bootstrap.cc:654] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:51.051116 5436 tablet_bootstrap.cc:492] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92: Bootstrap starting.
I20260927 11:11:51.051499 5435 log.cc:824] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:51.051774 5436 tablet_bootstrap.cc:654] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:51.052047 5436 log.cc:824] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:51.052150 5435 tablet_bootstrap.cc:492] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: No bootstrap required, opened a new log
I20260927 11:11:51.052199 5435 ts_tablet_manager.cc:1397] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 11:11:51.052798 5436 tablet_bootstrap.cc:492] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92: No bootstrap required, opened a new log
I20260927 11:11:51.052794 5434 tablet_bootstrap.cc:492] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: No bootstrap required, opened a new log
I20260927 11:11:51.052858 5436 ts_tablet_manager.cc:1397] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 11:11:51.052858 5434 ts_tablet_manager.cc:1397] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 11:11:51.054062 5435 raft_consensus.cc:348] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.054185 5435 raft_consensus.cc:374] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:51.054206 5435 raft_consensus.cc:729] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf687fb099e411a883ad3a67735a111, State: Initialized, Role: FOLLOWER
I20260927 11:11:51.054322 5435 consensus_queue.cc:260] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.054585 5420 heartbeater.cc:503] Master 127.17.95.254:32971 was elected leader, sending a full tablet report...
I20260927 11:11:51.054787 5435 ts_tablet_manager.cc:1428] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 11:11:51.055344 5436 raft_consensus.cc:348] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.055413 5434 raft_consensus.cc:348] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.055501 5436 raft_consensus.cc:374] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:51.055540 5434 raft_consensus.cc:374] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:51.055539 5436 raft_consensus.cc:729] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b373dde4b544ed83b5913cbb728e92, State: Initialized, Role: FOLLOWER
I20260927 11:11:51.055583 5434 raft_consensus.cc:729] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b31505fb4d94cf8be938b71b2159100, State: Initialized, Role: FOLLOWER
I20260927 11:11:51.055687 5434 consensus_queue.cc:260] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.055687 5436 consensus_queue.cc:260] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.055930 5436 ts_tablet_manager.cc:1428] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:51.055930 5434 ts_tablet_manager.cc:1428] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 11:11:51.055986 5289 heartbeater.cc:503] Master 127.17.95.254:32971 was elected leader, sending a full tablet report...
I20260927 11:11:51.056042 5158 heartbeater.cc:503] Master 127.17.95.254:32971 was elected leader, sending a full tablet report...
I20260927 11:11:51.058516 5440 raft_consensus.cc:482] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:11:51.058579 5440 raft_consensus.cc:504] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.058882 5440 leader_election.cc:288] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b373dde4b544ed83b5913cbb728e92 (127.17.95.193:44777), 0b31505fb4d94cf8be938b71b2159100 (127.17.95.194:34481)
I20260927 11:11:51.062530 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.059223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:44519 (local address 127.17.95.194:34481)
0927 11:11:51.059261 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.059262 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.059274 (+ 12us) server_negotiation.cc:407] Connection header received
0927 11:11:51.059328 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.059329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.059338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.059399 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:51.059549 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.060262 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.061682 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.062202 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.062270 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.062278 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.062437 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.062471 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:11:51.062530 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.059257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.95.195:41301 (local address 127.17.95.193:44777)
0927 11:11:51.059299 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.059301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.059308 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:11:51.059328 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.059328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.059338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.059395 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 11:11:51.059549 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.060251 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.061681 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.062199 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.062270 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.062278 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.062437 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.062470 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 11:11:51.062551 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.059049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44777 (local address 127.17.95.195:41301)
0927 11:11:51.059121 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.059133 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:51.059179 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:51.059397 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:51.059398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:51.059400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:51.059480 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.059481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.060349 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.060351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.061574 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.061576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.062277 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.062278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.062344 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.062347 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 11:11:51.062418 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 11:11:51.062467 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":70}
I20260927 11:11:51.062562 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.059049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.194:34481 (local address 127.17.95.195:44519)
0927 11:11:51.059230 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.059240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:11:51.059280 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:11:51.059402 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:11:51.059402 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:11:51.059403 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 11:11:51.059480 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.059481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.060349 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.060351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.061574 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:11:51.061576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.062278 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.062279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:11:51.062344 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.062348 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 11:11:51.062418 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 11:11:51.062467 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":93,"thread_start_us":89,"threads_started":1}
I20260927 11:11:51.062868 5112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571390c3d446408a8e9e553ec2419918" candidate_uuid: "6cf687fb099e411a883ad3a67735a111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b373dde4b544ed83b5913cbb728e92" is_pre_election: true
I20260927 11:11:51.063000 5112 raft_consensus.cc:2454] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf687fb099e411a883ad3a67735a111 in term 0.
I20260927 11:11:51.063217 5308 leader_election.cc:302] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b373dde4b544ed83b5913cbb728e92, 6cf687fb099e411a883ad3a67735a111; no voters:
I20260927 11:11:51.063334 5440 raft_consensus.cc:2781] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 11:11:51.063401 5440 raft_consensus.cc:482] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 11:11:51.063345 5243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571390c3d446408a8e9e553ec2419918" candidate_uuid: "6cf687fb099e411a883ad3a67735a111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b31505fb4d94cf8be938b71b2159100" is_pre_election: true
I20260927 11:11:51.063436 5440 raft_consensus.cc:3037] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:51.063475 5243 raft_consensus.cc:2454] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf687fb099e411a883ad3a67735a111 in term 0.
I20260927 11:11:51.064087 5440 raft_consensus.cc:504] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.064234 5440 leader_election.cc:288] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [CANDIDATE]: Term 1 election: Requested vote from peers 23b373dde4b544ed83b5913cbb728e92 (127.17.95.193:44777), 0b31505fb4d94cf8be938b71b2159100 (127.17.95.194:34481)
I20260927 11:11:51.064366 5112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571390c3d446408a8e9e553ec2419918" candidate_uuid: "6cf687fb099e411a883ad3a67735a111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b373dde4b544ed83b5913cbb728e92"
I20260927 11:11:51.064373 5243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571390c3d446408a8e9e553ec2419918" candidate_uuid: "6cf687fb099e411a883ad3a67735a111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b31505fb4d94cf8be938b71b2159100"
I20260927 11:11:51.064432 5112 raft_consensus.cc:3037] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:51.064481 5243 raft_consensus.cc:3037] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:51.065102 5112 raft_consensus.cc:2454] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf687fb099e411a883ad3a67735a111 in term 1.
I20260927 11:11:51.065356 5243 raft_consensus.cc:2454] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf687fb099e411a883ad3a67735a111 in term 1.
I20260927 11:11:51.065346 5308 leader_election.cc:302] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b373dde4b544ed83b5913cbb728e92, 6cf687fb099e411a883ad3a67735a111; no voters:
I20260927 11:11:51.065429 5440 raft_consensus.cc:2781] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:51.065587 5440 raft_consensus.cc:686] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 1 LEADER]: Becoming Leader. State: Replica: 6cf687fb099e411a883ad3a67735a111, State: Running, Role: LEADER
I20260927 11:11:51.065696 5440 consensus_queue.cc:237] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } }
I20260927 11:11:51.066388 4970 catalog_manager.cc:5495] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cf687fb099e411a883ad3a67735a111 (127.17.95.195). New cstate: current_term: 1 leader_uuid: "6cf687fb099e411a883ad3a67735a111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf687fb099e411a883ad3a67735a111" member_type: VOTER last_known_addr { host: "127.17.95.195" port: 44347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 } health_report { overall_health: UNKNOWN } } }
I20260927 11:11:51.074527 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.072202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46972 (local address 127.17.95.195:44347)
0927 11:11:51.072240 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:51.072242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:51.072286 (+ 44us) server_negotiation.cc:407] Connection header received
0927 11:11:51.072366 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:51.072366 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:51.072381 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:51.072470 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:11:51.072563 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.073011 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.073754 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:51.073976 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:51.074046 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:51.074205 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:11:51.074306 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:11:51.074395 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:51.074414 (+ 19us) server_negotiation.cc:299] Negotiation successful
0927 11:11:51.074432 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:11:51.076103 5112 raft_consensus.cc:1261] T 571390c3d446408a8e9e553ec2419918 P 23b373dde4b544ed83b5913cbb728e92 [term 1 FOLLOWER]: Refusing update from remote peer 6cf687fb099e411a883ad3a67735a111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:51.076143 5243 raft_consensus.cc:1261] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 1 FOLLOWER]: Refusing update from remote peer 6cf687fb099e411a883ad3a67735a111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:11:51.076419 5440 consensus_queue.cc:1035] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b373dde4b544ed83b5913cbb728e92" member_type: VOTER last_known_addr { host: "127.17.95.193" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:51.076555 5444 consensus_queue.cc:1035] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b31505fb4d94cf8be938b71b2159100" member_type: VOTER last_known_addr { host: "127.17.95.194" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:11:51.078660 5447 mvcc.cc:205] Tried to move back new op lower bound from 7333918765365645312 to 7333918765325717504. Current Snapshot: MvccSnapshot[applied={T|T < 7333918765365645312}]
I20260927 11:11:51.078837 5449 mvcc.cc:205] Tried to move back new op lower bound from 7333918765365645312 to 7333918765325717504. Current Snapshot: MvccSnapshot[applied={T|T < 7333918765365645312}]
I20260927 11:11:51.078842 5448 mvcc.cc:205] Tried to move back new op lower bound from 7333918765365645312 to 7333918765325717504. Current Snapshot: MvccSnapshot[applied={T|T < 7333918765365645312}]
I20260927 11:11:51.081079 4970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:51.081164 4970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 11:11:51.081897 4970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc3abf0af4b8445383e25d77fa1561d0: Sending DeleteTablet for 3 replicas of tablet 571390c3d446408a8e9e553ec2419918
I20260927 11:11:51.082234 5354 tablet_service.cc:1514] Processing DeleteTablet for tablet 571390c3d446408a8e9e553ec2419918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:51 UTC) from {username='slave'} at 127.0.0.1:46968
I20260927 11:11:51.082239 5223 tablet_service.cc:1514] Processing DeleteTablet for tablet 571390c3d446408a8e9e553ec2419918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:51 UTC) from {username='slave'} at 127.0.0.1:34096
I20260927 11:11:51.082427 5458 tablet_replica.cc:330] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: stopping tablet replica
I20260927 11:11:51.082453 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet 571390c3d446408a8e9e553ec2419918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 11:11:51 UTC) from {username='slave'} at 127.0.0.1:51650
I20260927 11:11:51.082508 5458 raft_consensus.cc:2229] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 11:11:51.082554 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5030
I20260927 11:11:51.082599 5458 raft_consensus.cc:2258] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:51.083554 5458 ts_tablet_manager.cc:1905] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:51.083794 5459 tablet_replica.cc:330] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: stopping tablet replica
I20260927 11:11:51.083962 5459 raft_consensus.cc:2229] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 1 LEADER]: Raft consensus shutting down.
I20260927 11:11:51.084116 5459 raft_consensus.cc:2258] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 11:11:51.084560 5459 ts_tablet_manager.cc:1905] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 11:11:51.085425 5458 ts_tablet_manager.cc:1918] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:51.085498 5458 log.cc:1196] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-1/wal/wals/571390c3d446408a8e9e553ec2419918
I20260927 11:11:51.085741 5458 ts_tablet_manager.cc:1939] T 571390c3d446408a8e9e553ec2419918 P 0b31505fb4d94cf8be938b71b2159100: Deleting consensus metadata
I20260927 11:11:51.086388 5459 ts_tablet_manager.cc:1918] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 11:11:51.086510 5459 log.cc:1196] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Deleting WAL directory at /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790507277271108-17791-0/minicluster-data/ts-2/wal/wals/571390c3d446408a8e9e553ec2419918
I20260927 11:11:51.086833 5459 ts_tablet_manager.cc:1939] T 571390c3d446408a8e9e553ec2419918 P 6cf687fb099e411a883ad3a67735a111: Deleting consensus metadata
I20260927 11:11:51.087347 4955 catalog_manager.cc:4841] TS 6cf687fb099e411a883ad3a67735a111 (127.17.95.195:44347): tablet 571390c3d446408a8e9e553ec2419918 (table test-table [id=5ac5a5bcc1a044a58d6b8ad6ffc1a001]) successfully deleted
I20260927 11:11:51.088169 4957 catalog_manager.cc:4841] TS 0b31505fb4d94cf8be938b71b2159100 (127.17.95.194:34481): tablet 571390c3d446408a8e9e553ec2419918 (table test-table [id=5ac5a5bcc1a044a58d6b8ad6ffc1a001]) successfully deleted
W20260927 11:11:51.089102 4955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.95.193:44777 (error 108)
I20260927 11:11:51.089246 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5161
I20260927 11:11:51.090139 5429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:51.090028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.193:44777 (local address 127.0.0.1:51652)
0927 11:11:51.090072 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 11:11:51.090109 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.193:44777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 11:11:51.090274 4955 catalog_manager.cc:4584] TS 23b373dde4b544ed83b5913cbb728e92 (127.17.95.193:44777): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 571390c3d446408a8e9e553ec2419918: Network error: Client connection negotiation failed: client connection to 127.17.95.193:44777: connect: Connection refused (error 111)
I20260927 11:11:51.095383 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5292
I20260927 11:11:51.099285 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 4939
2026-09-27T11:11:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (628 ms)
[----------] 6 tests from EncryptionPolicyTest (7875 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:51 dist-test-slave-6j0v krb5kdc[5464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:51 dist-test-slave-6j0v krb5kdc[5464](info): set up 2 sockets
Sep 27 11:11:51 dist-test-slave-6j0v krb5kdc[5464](info): commencing operation
krb5kdc: starting...
W20260927 11:11:52.102797 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:52 dist-test-slave-6j0v krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:52.213073 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:52.289988 5477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:52.290163 5477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:52.290184 5477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:52.291301 5477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:52.291347 5477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:52.291359 5477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:52.291370 5477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:52.291437 5477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:52.292834 5477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:52.293058 5477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:52.293284 5477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:52.298352 5482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:52.298444 5477 server_base.cc:1034] running on GCE node
W20260927 11:11:52.298375 5483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:52.298360 5485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:52.298779 5477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:52.299052 5477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:52.300251 5477 hybrid_clock.cc:648] HybridClock initialized: now 1790507512300236 us; error 31 us; skew 500 ppm
Sep 27 11:11:52 dist-test-slave-6j0v krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:52.302541 5477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:52.302902 5477 webserver.cc:460] Webserver started at http://127.0.0.1:39211/ using document root <none> and password file <none>
I20260927 11:11:52.303114 5477 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:52.303169 5477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:52.303256 5477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:52.304208 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "807a174dc21347e6bbdae2a6669fd8c3"
format_stamp: "Formatted at 2026-09-27 11:11:52 on dist-test-slave-6j0v"
server_key: "e518a9a462ec1c3fae21ffe2480495e0"
server_key_iv: "f60abddbbaab248b3a2fe517770d83da"
server_key_version: "encryptionkey@0"
I20260927 11:11:52.304489 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "807a174dc21347e6bbdae2a6669fd8c3"
format_stamp: "Formatted at 2026-09-27 11:11:52 on dist-test-slave-6j0v"
server_key: "e518a9a462ec1c3fae21ffe2480495e0"
server_key_iv: "f60abddbbaab248b3a2fe517770d83da"
server_key_version: "encryptionkey@0"
I20260927 11:11:52.305698 5477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 11:11:52.306310 5492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:52.306459 5477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:52.306540 5477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "807a174dc21347e6bbdae2a6669fd8c3"
format_stamp: "Formatted at 2026-09-27 11:11:52 on dist-test-slave-6j0v"
server_key: "e518a9a462ec1c3fae21ffe2480495e0"
server_key_iv: "f60abddbbaab248b3a2fe517770d83da"
server_key_version: "encryptionkey@0"
I20260927 11:11:52.306591 5477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:52.319377 5477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:52.321473 5477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:52.321571 5477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:52.325131 5477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33071
I20260927 11:11:52.325183 5544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33071 every 8 connection(s)
I20260927 11:11:52.325495 5477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:52.326306 5545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:52.326843 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5477
I20260927 11:11:52.326961 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:52.327188 17791 external_mini_cluster.cc:1428] Setting key cf32838e48c63615840bd5c8622ebfca
I20260927 11:11:52.328930 5545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3: Bootstrap starting.
I20260927 11:11:52.329497 5545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:52.329774 5545 log.cc:824] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:52.330457 5545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3: No bootstrap required, opened a new log
I20260927 11:11:52.332213 5545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807a174dc21347e6bbdae2a6669fd8c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33071 } }
I20260927 11:11:52.332356 5545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:52.332401 5545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807a174dc21347e6bbdae2a6669fd8c3, State: Initialized, Role: FOLLOWER
I20260927 11:11:52.332527 5545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807a174dc21347e6bbdae2a6669fd8c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33071 } }
I20260927 11:11:52.332614 5545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:52.332655 5545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:52.332700 5545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:52.333177 5545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807a174dc21347e6bbdae2a6669fd8c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33071 } }
I20260927 11:11:52.333303 5545 leader_election.cc:302] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 807a174dc21347e6bbdae2a6669fd8c3; no voters:
I20260927 11:11:52.333482 5545 leader_election.cc:288] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:52.333506 5550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:52.333612 5550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [term 1 LEADER]: Becoming Leader. State: Replica: 807a174dc21347e6bbdae2a6669fd8c3, State: Running, Role: LEADER
I20260927 11:11:52.333707 5550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807a174dc21347e6bbdae2a6669fd8c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33071 } }
Sep 27 11:11:52 dist-test-slave-6j0v krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 11:11:52.333743 5545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:52.334028 5550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 807a174dc21347e6bbdae2a6669fd8c3. Latest consensus state: current_term: 1 leader_uuid: "807a174dc21347e6bbdae2a6669fd8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807a174dc21347e6bbdae2a6669fd8c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33071 } } }
I20260927 11:11:52.334097 5550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:52.334044 5551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "807a174dc21347e6bbdae2a6669fd8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807a174dc21347e6bbdae2a6669fd8c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33071 } } }
I20260927 11:11:52.334271 5551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3 [sys.catalog]: This master's current role is: LEADER
W20260927 11:11:52.335551 5565 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:11:52.335604 5565 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:11:52.335673 5555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:52.335805 5555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:52.337779 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:52.327733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52043 (local address 127.0.0.1:33071)
0927 11:11:52.327913 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:52.327922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:52.327953 (+ 31us) server_negotiation.cc:407] Connection header received
0927 11:11:52.328346 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:52.328352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:52.328402 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:52.328542 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:52.329229 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:52.330340 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:52.331180 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:52.331603 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:52.331741 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:52.334273 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:52.334277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:52.334277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:52.334288 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:52.336227 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:52.336659 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:52.336662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:52.336663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:52.336769 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:52.337015 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:52.337016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:52.337017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:52.337247 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:52.337422 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:52.337511 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 11:11:52.337607 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
I20260927 11:11:52.338776 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:52.344950 5555 catalog_manager.cc:1347] Generated new cluster ID: 644bb359c7cf4e3185aab563f24c273c
I20260927 11:11:52.344997 5555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:52.346815 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:52.340141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46161 (local address 127.0.0.1:33071)
0927 11:11:52.340174 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:52.340176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:52.340221 (+ 45us) server_negotiation.cc:407] Connection header received
0927 11:11:52.340288 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:52.340289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:52.340296 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:52.340334 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:52.340914 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:52.341596 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:52.342549 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:52.342741 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:52.342790 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:52.344033 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:52.344034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:52.344034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:52.344039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:52.345631 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:52.346053 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:52.346054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:52.346054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:52.346108 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:52.346326 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:52.346327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:52.346327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:52.346436 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:52.346520 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:52.346676 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 11:11:52.346731 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:11:52.362524 5555 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:52.363005 5555 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:52.364923 5555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 807a174dc21347e6bbdae2a6669fd8c3: Generated new TSK 0
I20260927 11:11:52.365090 5555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:52.373479 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5477
W20260927 11:11:52.377727 5462 connection.cc:535] client connection to 127.0.0.1:33071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33071): Connection reset by peer (error 104)
2026-09-27T11:11:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:52 dist-test-slave-6j0v krb5kdc[5576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:52 dist-test-slave-6j0v krb5kdc[5576](info): set up 2 sockets
Sep 27 11:11:52 dist-test-slave-6j0v krb5kdc[5576](info): commencing operation
krb5kdc: starting...
W20260927 11:11:53.375159 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:53 dist-test-slave-6j0v krb5kdc[5576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:53.485891 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:53.564221 5589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:53.564405 5589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:53.564426 5589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:53.565485 5589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:53.565519 5589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:53.565532 5589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:53.565546 5589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:53.565557 5589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:53.566907 5589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38311
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:53.567121 5589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:53.567303 5589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:53.571847 5595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:53.571862 5597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:53.572041 5589 server_base.cc:1034] running on GCE node
W20260927 11:11:53.572140 5594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:53.572405 5589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:53.572675 5589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:53.573859 5589 hybrid_clock.cc:648] HybridClock initialized: now 1790507513573846 us; error 34 us; skew 500 ppm
Sep 27 11:11:53 dist-test-slave-6j0v krb5kdc[5576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507513, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:53.580520 5589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:53.580924 5589 webserver.cc:460] Webserver started at http://127.0.0.1:41181/ using document root <none> and password file <none>
I20260927 11:11:53.581144 5589 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:53.581207 5589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:53.581370 5589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:53.582324 5589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "d686221caae7441d9166184e0735e00e"
format_stamp: "Formatted at 2026-09-27 11:11:53 on dist-test-slave-6j0v"
server_key: "cfb0beb34940c1caebab0827bc617ebe"
server_key_iv: "f954cb04ab8024f77de213c27d118d04"
server_key_version: "encryptionkey@0"
I20260927 11:11:53.582685 5589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "d686221caae7441d9166184e0735e00e"
format_stamp: "Formatted at 2026-09-27 11:11:53 on dist-test-slave-6j0v"
server_key: "cfb0beb34940c1caebab0827bc617ebe"
server_key_iv: "f954cb04ab8024f77de213c27d118d04"
server_key_version: "encryptionkey@0"
I20260927 11:11:53.584057 5589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:11:53.584756 5604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:53.584926 5589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:53.585017 5589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "d686221caae7441d9166184e0735e00e"
format_stamp: "Formatted at 2026-09-27 11:11:53 on dist-test-slave-6j0v"
server_key: "cfb0beb34940c1caebab0827bc617ebe"
server_key_iv: "f954cb04ab8024f77de213c27d118d04"
server_key_version: "encryptionkey@0"
I20260927 11:11:53.585114 5589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:53.592672 5589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:53.594806 5589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:53.594959 5589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:53.598784 5589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38311
I20260927 11:11:53.598814 5656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38311 every 8 connection(s)
I20260927 11:11:53.599200 5589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:53.600149 5657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:53.600232 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5589
I20260927 11:11:53.600340 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:53.600576 17791 external_mini_cluster.cc:1428] Setting key e59a9499636aebe0c181220d964b5494
I20260927 11:11:53.603511 5657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e: Bootstrap starting.
I20260927 11:11:53.604262 5657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:53.604557 5657 log.cc:824] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:53.605254 5657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e: No bootstrap required, opened a new log
I20260927 11:11:53.606240 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:53.601094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49831 (local address 127.0.0.1:38311)
0927 11:11:53.601265 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:53.601272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:53.601296 (+ 24us) server_negotiation.cc:407] Connection header received
0927 11:11:53.601776 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:53.601782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:53.601828 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:53.601937 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:53.602238 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:53.603188 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:53.604381 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:53.604845 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:53.604964 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:53.605680 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:53.605683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:53.605683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:53.605832 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:53.605859 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:53.605870 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:53.605961 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:53.606041 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 11:11:53.606067 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260927 11:11:53.607479 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:53.607734 5657 raft_consensus.cc:348] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d686221caae7441d9166184e0735e00e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260927 11:11:53.607869 5657 raft_consensus.cc:374] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:53.607894 5657 raft_consensus.cc:729] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d686221caae7441d9166184e0735e00e, State: Initialized, Role: FOLLOWER
I20260927 11:11:53.608034 5657 consensus_queue.cc:260] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d686221caae7441d9166184e0735e00e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260927 11:11:53.608109 5657 raft_consensus.cc:388] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:53.608152 5657 raft_consensus.cc:482] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:53.608206 5657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:53.608857 5657 raft_consensus.cc:504] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d686221caae7441d9166184e0735e00e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260927 11:11:53.609015 5657 leader_election.cc:302] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d686221caae7441d9166184e0735e00e; no voters:
I20260927 11:11:53.609189 5657 leader_election.cc:288] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:53.609220 5667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:53.609359 5667 raft_consensus.cc:686] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [term 1 LEADER]: Becoming Leader. State: Replica: d686221caae7441d9166184e0735e00e, State: Running, Role: LEADER
I20260927 11:11:53.609452 5657 sys_catalog.cc:564] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:53.609508 5667 consensus_queue.cc:237] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d686221caae7441d9166184e0735e00e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260927 11:11:53.609875 5667 sys_catalog.cc:455] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d686221caae7441d9166184e0735e00e. Latest consensus state: current_term: 1 leader_uuid: "d686221caae7441d9166184e0735e00e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d686221caae7441d9166184e0735e00e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } } }
I20260927 11:11:53.609881 5668 sys_catalog.cc:455] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d686221caae7441d9166184e0735e00e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d686221caae7441d9166184e0735e00e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } } }
I20260927 11:11:53.610076 5668 sys_catalog.cc:458] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:53.610308 5667 sys_catalog.cc:458] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:53.610591 5676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:53.611148 5676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:53.612514 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:53.608906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55299 (local address 127.0.0.1:38311)
0927 11:11:53.608939 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:53.608941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:53.608995 (+ 54us) server_negotiation.cc:407] Connection header received
0927 11:11:53.609078 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:53.609078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:53.609101 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:53.609156 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:53.609383 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:53.610011 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:53.611060 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:53.611355 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:53.611396 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:53.612241 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:53.612243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:53.612243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:53.612247 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:53.612260 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:53.612265 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:53.612295 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:53.612445 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 11:11:53.612456 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:11:53.612891 5676 catalog_manager.cc:1347] Generated new cluster ID: 69028abe8db74749b61b8945161fa944
I20260927 11:11:53.612947 5676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:53.619561 5676 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:53.620010 5676 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:53.628511 5676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d686221caae7441d9166184e0735e00e: Generated new TSK 0
I20260927 11:11:53.628675 5676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:53.631392 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5589
W20260927 11:11:53.635480 5574 connection.cc:535] client connection to 127.0.0.1:38311 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38311): Connection reset by peer (error 104)
2026-09-27T11:11:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1257 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:53 dist-test-slave-6j0v krb5kdc[5688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:53 dist-test-slave-6j0v krb5kdc[5688](info): set up 2 sockets
Sep 27 11:11:53 dist-test-slave-6j0v krb5kdc[5688](info): commencing operation
krb5kdc: starting...
W20260927 11:11:54.635160 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:54 dist-test-slave-6j0v krb5kdc[5688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:54.746219 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39731
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:54.824075 5701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:54.824316 5701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:54.824354 5701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:54.825455 5701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:54.825529 5701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:54.825569 5701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:54.825590 5701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:54.825621 5701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:54.827044 5701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39731
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:54.827316 5701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:54.827569 5701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:54.832333 5707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:54.832361 5709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:54.832530 5701 server_base.cc:1034] running on GCE node
W20260927 11:11:54.832361 5706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:54.832911 5701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:54.833186 5701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:54.834347 5701 hybrid_clock.cc:648] HybridClock initialized: now 1790507514834346 us; error 38 us; skew 500 ppm
Sep 27 11:11:54 dist-test-slave-6j0v krb5kdc[5688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:54.836823 5701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:54.837163 5701 webserver.cc:460] Webserver started at http://127.0.0.1:36149/ using document root <none> and password file <none>
I20260927 11:11:54.837342 5701 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:54.837379 5701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:54.837476 5701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:54.838392 5701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "7b5a307ebb47488cbbaacf84db668420"
format_stamp: "Formatted at 2026-09-27 11:11:54 on dist-test-slave-6j0v"
server_key: "2815aceda7920b1bb2794a4fe01af4b1"
server_key_iv: "f619147d6ec32c83be6e3dd3c046b236"
server_key_version: "encryptionkey@0"
I20260927 11:11:54.838665 5701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "7b5a307ebb47488cbbaacf84db668420"
format_stamp: "Formatted at 2026-09-27 11:11:54 on dist-test-slave-6j0v"
server_key: "2815aceda7920b1bb2794a4fe01af4b1"
server_key_iv: "f619147d6ec32c83be6e3dd3c046b236"
server_key_version: "encryptionkey@0"
I20260927 11:11:54.840005 5701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:54.840720 5716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:54.840906 5701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:54.841002 5701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "7b5a307ebb47488cbbaacf84db668420"
format_stamp: "Formatted at 2026-09-27 11:11:54 on dist-test-slave-6j0v"
server_key: "2815aceda7920b1bb2794a4fe01af4b1"
server_key_iv: "f619147d6ec32c83be6e3dd3c046b236"
server_key_version: "encryptionkey@0"
I20260927 11:11:54.841069 5701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:54.844304 5701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:54.846166 5701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:54.846313 5701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:54.850265 5701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39731
I20260927 11:11:54.850291 5768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39731 every 8 connection(s)
I20260927 11:11:54.850589 5701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:54.851449 5769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:54.853509 5769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420: Bootstrap starting.
I20260927 11:11:54.854085 5769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:54.854382 5769 log.cc:824] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:54.855139 5769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420: No bootstrap required, opened a new log
I20260927 11:11:54.856705 5769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5a307ebb47488cbbaacf84db668420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39731 } }
I20260927 11:11:54.856844 5769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:54.856866 5769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b5a307ebb47488cbbaacf84db668420, State: Initialized, Role: FOLLOWER
I20260927 11:11:54.857019 5769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5a307ebb47488cbbaacf84db668420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39731 } }
I20260927 11:11:54.857095 5769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:54.857137 5769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:54.857193 5769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:54.857688 5769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5a307ebb47488cbbaacf84db668420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39731 } }
I20260927 11:11:54.857796 5769 leader_election.cc:302] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b5a307ebb47488cbbaacf84db668420; no voters:
I20260927 11:11:54.857971 5769 leader_election.cc:288] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:54.858027 5772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:54.858206 5772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [term 1 LEADER]: Becoming Leader. State: Replica: 7b5a307ebb47488cbbaacf84db668420, State: Running, Role: LEADER
I20260927 11:11:54.858265 5769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:54.858341 5772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5a307ebb47488cbbaacf84db668420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39731 } }
I20260927 11:11:54.858713 5774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b5a307ebb47488cbbaacf84db668420. Latest consensus state: current_term: 1 leader_uuid: "7b5a307ebb47488cbbaacf84db668420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5a307ebb47488cbbaacf84db668420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39731 } } }
I20260927 11:11:54.858800 5774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:54.858918 5773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b5a307ebb47488cbbaacf84db668420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b5a307ebb47488cbbaacf84db668420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39731 } } }
I20260927 11:11:54.858981 5773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:54.859123 5781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:54.859644 5781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:54.860296 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5701
I20260927 11:11:54.860432 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:54.860613 17791 external_mini_cluster.cc:1428] Setting key 023f86c78db8213198536065ca30de9b
I20260927 11:11:54.861289 5781 catalog_manager.cc:1347] Generated new cluster ID: 987e604320ed471ebc4a1f1e411bbec6
I20260927 11:11:54.861339 5781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:54.864719 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:54.861131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43895 (local address 127.0.0.1:39731)
0927 11:11:54.863342 (+ 2211us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:54.863349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:54.863372 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:11:54.863465 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:54.863471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:54.863493 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:54.863626 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:54.864315 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:54.864318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:54.864318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:54.864322 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:54.864354 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:54.864370 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:54.864458 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:54.864529 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 11:11:54.864569 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2169,"thread_start_us":71,"threads_started":1}
I20260927 11:11:54.865561 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:54.872254 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:54.867421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47889 (local address 127.0.0.1:39731)
0927 11:11:54.867456 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:54.867458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:54.871254 (+ 3796us) server_negotiation.cc:407] Connection header received
0927 11:11:54.871328 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:54.871329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:54.871344 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:54.871416 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:54.872034 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:54.872035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:54.872035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:54.872039 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:54.872052 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:54.872056 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:54.872099 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:54.872167 (+ 68us) server_negotiation.cc:299] Negotiation successful
0927 11:11:54.872181 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:11:54.879196 5781 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:54.879341 5781 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:54.889112 5781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b5a307ebb47488cbbaacf84db668420: Generated new TSK 0
I20260927 11:11:54.889279 5781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:54.893932 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5701
2026-09-27T11:11:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:54 dist-test-slave-6j0v krb5kdc[5800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:54 dist-test-slave-6j0v krb5kdc[5800](info): set up 2 sockets
Sep 27 11:11:54 dist-test-slave-6j0v krb5kdc[5800](info): commencing operation
krb5kdc: starting...
W20260927 11:11:55.898958 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:55 dist-test-slave-6j0v krb5kdc[5800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:56.008016 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:56.085896 5813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:56.086077 5813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:56.086099 5813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:56.087229 5813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:56.087266 5813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:56.087280 5813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:56.087292 5813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:56.087304 5813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:56.088706 5813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:56.088872 5813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:56.089028 5813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:56.093575 5819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:56.093640 5818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:56.093628 5821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:56.093881 5813 server_base.cc:1034] running on GCE node
I20260927 11:11:56.094141 5813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:56.094398 5813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:56.095556 5813 hybrid_clock.cc:648] HybridClock initialized: now 1790507516095537 us; error 41 us; skew 500 ppm
Sep 27 11:11:56 dist-test-slave-6j0v krb5kdc[5800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507516, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:56.098028 5813 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:56.098429 5813 webserver.cc:460] Webserver started at http://127.0.0.1:40355/ using document root <none> and password file <none>
I20260927 11:11:56.098661 5813 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:56.098727 5813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:56.098872 5813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:56.099893 5813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "0d2abb5e28d0410281cd0a702bab24ac"
format_stamp: "Formatted at 2026-09-27 11:11:56 on dist-test-slave-6j0v"
server_key: "73e58ec0425e9eb2c57ae18c23207cc4"
server_key_iv: "24b294e56a09b7eb16a6811e90e53b53"
server_key_version: "encryptionkey@0"
I20260927 11:11:56.100239 5813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "0d2abb5e28d0410281cd0a702bab24ac"
format_stamp: "Formatted at 2026-09-27 11:11:56 on dist-test-slave-6j0v"
server_key: "73e58ec0425e9eb2c57ae18c23207cc4"
server_key_iv: "24b294e56a09b7eb16a6811e90e53b53"
server_key_version: "encryptionkey@0"
I20260927 11:11:56.101614 5813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:56.102314 5828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:56.102491 5813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:56.102593 5813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "0d2abb5e28d0410281cd0a702bab24ac"
format_stamp: "Formatted at 2026-09-27 11:11:56 on dist-test-slave-6j0v"
server_key: "73e58ec0425e9eb2c57ae18c23207cc4"
server_key_iv: "24b294e56a09b7eb16a6811e90e53b53"
server_key_version: "encryptionkey@0"
I20260927 11:11:56.102676 5813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:56.114725 5813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:56.116658 5813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:56.116828 5813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:56.120653 5813 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41033
I20260927 11:11:56.120693 5880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41033 every 8 connection(s)
I20260927 11:11:56.121026 5813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:56.121991 5881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:56.121997 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5813
I20260927 11:11:56.122115 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:56.122334 17791 external_mini_cluster.cc:1428] Setting key 59cfa4ea6874b498ef50cba6090a56ee
I20260927 11:11:56.125248 5881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac: Bootstrap starting.
I20260927 11:11:56.125946 5881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:56.126240 5881 log.cc:824] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:56.126953 5881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac: No bootstrap required, opened a new log
Sep 27 11:11:56 dist-test-slave-6j0v krb5kdc[5800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 11:11:56.129613 5881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2abb5e28d0410281cd0a702bab24ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260927 11:11:56.129740 5881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:56.129761 5881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2abb5e28d0410281cd0a702bab24ac, State: Initialized, Role: FOLLOWER
I20260927 11:11:56.129891 5881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2abb5e28d0410281cd0a702bab24ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260927 11:11:56.129949 5881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:56.129971 5881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:56.130034 5881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:56.130573 5881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2abb5e28d0410281cd0a702bab24ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260927 11:11:56.130687 5881 leader_election.cc:302] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d2abb5e28d0410281cd0a702bab24ac; no voters:
I20260927 11:11:56.130852 5881 leader_election.cc:288] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:56.130883 5886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:56.130996 5886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [term 1 LEADER]: Becoming Leader. State: Replica: 0d2abb5e28d0410281cd0a702bab24ac, State: Running, Role: LEADER
I20260927 11:11:56.131131 5881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:56.131132 5886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2abb5e28d0410281cd0a702bab24ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260927 11:11:56.132463 5888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d2abb5e28d0410281cd0a702bab24ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2abb5e28d0410281cd0a702bab24ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } } }
I20260927 11:11:56.132545 5888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:56.132937 5887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d2abb5e28d0410281cd0a702bab24ac. Latest consensus state: current_term: 1 leader_uuid: "0d2abb5e28d0410281cd0a702bab24ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2abb5e28d0410281cd0a702bab24ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } } }
W20260927 11:11:56.132956 5901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260927 11:11:56.133036 5887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac [sys.catalog]: This master's current role is: LEADER
W20260927 11:11:56.133049 5901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:11:56.133131 5903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:56.133273 5903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:56.134219 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:56.122890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56657 (local address 127.0.0.1:41033)
0927 11:11:56.123100 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:56.123109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:56.123141 (+ 32us) server_negotiation.cc:407] Connection header received
0927 11:11:56.123593 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:56.123599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:56.123653 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:56.123764 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:56.124563 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:56.125490 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:56.126573 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:56.127147 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:56.127240 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:56.130101 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:56.130106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:56.130106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:56.130118 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:56.132162 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:56.132656 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:56.132659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:56.132659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:56.132759 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:56.132974 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:56.132975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:56.132975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:56.133291 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:56.133415 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:56.133879 (+ 464us) server_negotiation.cc:299] Negotiation successful
0927 11:11:56.134015 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260927 11:11:56.135131 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:56.142925 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:56.136324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58843 (local address 127.0.0.1:41033)
0927 11:11:56.136359 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:56.136361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:56.136431 (+ 70us) server_negotiation.cc:407] Connection header received
0927 11:11:56.136493 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:56.136493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:56.136500 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:56.136543 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:56.137163 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:56.137594 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:56.138537 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:56.138937 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:56.139017 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:56.140171 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:56.140172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:56.140173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:56.140179 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:56.141882 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:56.142212 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:56.142213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:56.142213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:56.142263 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:56.142461 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:56.142462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:56.142462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:56.142568 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:56.142674 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:56.142766 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 11:11:56.142834 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:11:56.143491 5903 catalog_manager.cc:1347] Generated new cluster ID: 21fcdcc3ff4f4a73a54155a6af68e2a2
I20260927 11:11:56.143558 5903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:56.169006 5903 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:56.169566 5903 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:56.173412 5903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d2abb5e28d0410281cd0a702bab24ac: Generated new TSK 0
I20260927 11:11:56.173588 5903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:56.183454 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5813
2026-09-27T11:11:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:56 dist-test-slave-6j0v krb5kdc[5912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:56 dist-test-slave-6j0v krb5kdc[5912](info): set up 2 sockets
Sep 27 11:11:56 dist-test-slave-6j0v krb5kdc[5912](info): commencing operation
krb5kdc: starting...
W20260927 11:11:57.191196 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:57 dist-test-slave-6j0v krb5kdc[5912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:57.305632 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33407
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:57.382064 5925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:57.382239 5925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:57.382261 5925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:57.383337 5925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:57.383375 5925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:57.383387 5925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:57.383400 5925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:57.383411 5925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:57.384819 5925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:57.385038 5925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:57.385238 5925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:57.389773 5931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:57.390007 5930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:57.390036 5933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:57.390090 5925 server_base.cc:1034] running on GCE node
I20260927 11:11:57.390385 5925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:57.390756 5925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:57.391939 5925 hybrid_clock.cc:648] HybridClock initialized: now 1790507517391922 us; error 34 us; skew 500 ppm
Sep 27 11:11:57 dist-test-slave-6j0v krb5kdc[5912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507517, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:57.394495 5925 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:57.394948 5925 webserver.cc:460] Webserver started at http://127.0.0.1:45039/ using document root <none> and password file <none>
I20260927 11:11:57.395216 5925 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:57.395280 5925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:57.395413 5925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:57.396374 5925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf"
format_stamp: "Formatted at 2026-09-27 11:11:57 on dist-test-slave-6j0v"
server_key: "1fe8540e65cfb117532c53a9d9d2c683"
server_key_iv: "ef48461e7092be1e22f46d3f62bcbc6d"
server_key_version: "encryptionkey@0"
I20260927 11:11:57.396751 5925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf"
format_stamp: "Formatted at 2026-09-27 11:11:57 on dist-test-slave-6j0v"
server_key: "1fe8540e65cfb117532c53a9d9d2c683"
server_key_iv: "ef48461e7092be1e22f46d3f62bcbc6d"
server_key_version: "encryptionkey@0"
I20260927 11:11:57.398038 5925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:57.398754 5940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:57.398916 5925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:11:57.399013 5925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf"
format_stamp: "Formatted at 2026-09-27 11:11:57 on dist-test-slave-6j0v"
server_key: "1fe8540e65cfb117532c53a9d9d2c683"
server_key_iv: "ef48461e7092be1e22f46d3f62bcbc6d"
server_key_version: "encryptionkey@0"
I20260927 11:11:57.399108 5925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:57.413077 5925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:57.415274 5925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:57.415375 5925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:57.419381 5925 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33407
I20260927 11:11:57.419413 5992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33407 every 8 connection(s)
I20260927 11:11:57.419744 5925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:57.420547 5993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:57.422812 5993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf: Bootstrap starting.
I20260927 11:11:57.423333 5993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:57.423564 5993 log.cc:824] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:57.424155 5993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf: No bootstrap required, opened a new log
I20260927 11:11:57.425638 5993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260927 11:11:57.425766 5993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:57.425788 5993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2baa3bad9b0946e3b8e4bc852e2fd5bf, State: Initialized, Role: FOLLOWER
I20260927 11:11:57.425917 5993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260927 11:11:57.425999 5993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:57.426023 5993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:57.426050 5993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:57.426494 5993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260927 11:11:57.426573 5993 leader_election.cc:302] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2baa3bad9b0946e3b8e4bc852e2fd5bf; no voters:
I20260927 11:11:57.426687 5993 leader_election.cc:288] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:57.426730 5996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:57.426883 5996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [term 1 LEADER]: Becoming Leader. State: Replica: 2baa3bad9b0946e3b8e4bc852e2fd5bf, State: Running, Role: LEADER
I20260927 11:11:57.426918 5993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:57.427021 5996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260927 11:11:57.427455 5998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2baa3bad9b0946e3b8e4bc852e2fd5bf. Latest consensus state: current_term: 1 leader_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } } }
I20260927 11:11:57.427574 5997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baa3bad9b0946e3b8e4bc852e2fd5bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } } }
I20260927 11:11:57.427601 5998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:57.427695 5997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:57.428160 6005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:57.428681 6005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:57.429940 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 5925
I20260927 11:11:57.430068 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:57.430276 17791 external_mini_cluster.cc:1428] Setting key 35c27e244fe59b3d79067983f3f8eca9
I20260927 11:11:57.430477 6005 catalog_manager.cc:1347] Generated new cluster ID: aa721cdf6ee745a0a49587beee9a0518
I20260927 11:11:57.430523 6005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:57.435994 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:57.430847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42503 (local address 127.0.0.1:33407)
0927 11:11:57.430994 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:57.431003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:57.431037 (+ 34us) server_negotiation.cc:407] Connection header received
0927 11:11:57.431173 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:57.431180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:57.431243 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:57.431375 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:57.431584 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:57.432897 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:57.433886 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:57.434480 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:57.434590 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:57.435334 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:57.435338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:57.435338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:57.435550 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:57.435591 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:57.435605 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:57.435672 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:57.435741 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 11:11:57.435800 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260927 11:11:57.436973 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:57.437106 6005 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:57.437834 6005 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:57.441130 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:57.438373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40511 (local address 127.0.0.1:33407)
0927 11:11:57.438406 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:57.438408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:57.438519 (+ 111us) server_negotiation.cc:407] Connection header received
0927 11:11:57.438588 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:57.438588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:57.438598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:57.438643 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:57.438742 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:57.439273 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:57.439986 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:57.440194 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:57.440267 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:57.440887 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:57.440888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:57.440888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:57.440892 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:57.440905 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:57.440909 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:57.440959 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:57.441016 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 11:11:57.441039 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:11:57.442467 6005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2baa3bad9b0946e3b8e4bc852e2fd5bf: Generated new TSK 0
I20260927 11:11:57.442615 6005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:57.445340 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 5925
2026-09-27T11:11:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1261 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:57 dist-test-slave-6j0v krb5kdc[6024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:57 dist-test-slave-6j0v krb5kdc[6024](info): set up 2 sockets
Sep 27 11:11:57 dist-test-slave-6j0v krb5kdc[6024](info): commencing operation
krb5kdc: starting...
W20260927 11:11:58.443145 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:58 dist-test-slave-6j0v krb5kdc[6024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:58.561439 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:58.638588 6037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:58.638770 6037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:58.638792 6037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:58.639876 6037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:58.639915 6037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:58.639930 6037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:58.639941 6037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:58.639952 6037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:58.641201 6037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:58.641366 6037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:58.641505 6037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:58.645890 6045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:58.645902 6042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:58.645936 6043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:58.646351 6037 server_base.cc:1034] running on GCE node
I20260927 11:11:58.646593 6037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:58.646873 6037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:58.648042 6037 hybrid_clock.cc:648] HybridClock initialized: now 1790507518648022 us; error 28 us; skew 500 ppm
Sep 27 11:11:58 dist-test-slave-6j0v krb5kdc[6024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507518, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:58.653705 6037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:58.654057 6037 webserver.cc:460] Webserver started at http://127.0.0.1:46105/ using document root <none> and password file <none>
I20260927 11:11:58.654278 6037 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:58.654321 6037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:58.654476 6037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:58.655561 6037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "d8b268b607c04ed8bef001b917c5b66c"
format_stamp: "Formatted at 2026-09-27 11:11:58 on dist-test-slave-6j0v"
server_key: "a79b781297399e8324cdbcc3cac2a96b"
server_key_iv: "a6639560562282417c4332fbd39f267e"
server_key_version: "encryptionkey@0"
I20260927 11:11:58.655920 6037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "d8b268b607c04ed8bef001b917c5b66c"
format_stamp: "Formatted at 2026-09-27 11:11:58 on dist-test-slave-6j0v"
server_key: "a79b781297399e8324cdbcc3cac2a96b"
server_key_iv: "a6639560562282417c4332fbd39f267e"
server_key_version: "encryptionkey@0"
I20260927 11:11:58.657196 6037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:11:58.657953 6052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:58.658157 6037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:58.658242 6037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "d8b268b607c04ed8bef001b917c5b66c"
format_stamp: "Formatted at 2026-09-27 11:11:58 on dist-test-slave-6j0v"
server_key: "a79b781297399e8324cdbcc3cac2a96b"
server_key_iv: "a6639560562282417c4332fbd39f267e"
server_key_version: "encryptionkey@0"
I20260927 11:11:58.658313 6037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:58.661545 6037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:58.663453 6037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:58.663580 6037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:58.667495 6037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34159
I20260927 11:11:58.667554 6104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34159 every 8 connection(s)
I20260927 11:11:58.667876 6037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:58.668691 6105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:58.670684 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c: Bootstrap starting.
I20260927 11:11:58.671267 6105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:58.671584 6105 log.cc:824] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:58.672224 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c: No bootstrap required, opened a new log
I20260927 11:11:58.673761 6105 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b268b607c04ed8bef001b917c5b66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34159 } }
I20260927 11:11:58.673885 6105 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:58.673906 6105 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b268b607c04ed8bef001b917c5b66c, State: Initialized, Role: FOLLOWER
I20260927 11:11:58.674046 6105 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b268b607c04ed8bef001b917c5b66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34159 } }
I20260927 11:11:58.674104 6105 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:58.674126 6105 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:58.674173 6105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:58.674654 6105 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b268b607c04ed8bef001b917c5b66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34159 } }
I20260927 11:11:58.674759 6105 leader_election.cc:302] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8b268b607c04ed8bef001b917c5b66c; no voters:
I20260927 11:11:58.674924 6105 leader_election.cc:288] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:58.674973 6108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:58.675171 6108 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [term 1 LEADER]: Becoming Leader. State: Replica: d8b268b607c04ed8bef001b917c5b66c, State: Running, Role: LEADER
I20260927 11:11:58.675256 6105 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:58.675304 6108 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b268b607c04ed8bef001b917c5b66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34159 } }
I20260927 11:11:58.675556 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6037
I20260927 11:11:58.675695 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:58.675654 6110 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8b268b607c04ed8bef001b917c5b66c. Latest consensus state: current_term: 1 leader_uuid: "d8b268b607c04ed8bef001b917c5b66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b268b607c04ed8bef001b917c5b66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34159 } } }
I20260927 11:11:58.675731 6110 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:58.675779 6109 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8b268b607c04ed8bef001b917c5b66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b268b607c04ed8bef001b917c5b66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34159 } } }
I20260927 11:11:58.675889 6109 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:58.675930 17791 external_mini_cluster.cc:1428] Setting key 8db15238bd13b4a90ee796e9e0e88341
I20260927 11:11:58.675961 6112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:58.676513 6112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:11:58.678287 6112 catalog_manager.cc:1347] Generated new cluster ID: 60fc279200bd4120aee5bdb8466b74a5
I20260927 11:11:58.678340 6112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:58.678865 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:58.676549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34415 (local address 127.0.0.1:34159)
0927 11:11:58.676713 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:58.676723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:58.676856 (+ 133us) server_negotiation.cc:407] Connection header received
0927 11:11:58.677197 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:58.677205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:58.677236 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:58.677366 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:58.678438 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:58.678442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:58.678442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:58.678447 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:58.678487 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:58.678501 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:58.678536 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:58.678673 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 11:11:58.678711 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":94,"threads_started":1}
I20260927 11:11:58.679852 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:58.682256 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:58.681193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51685 (local address 127.0.0.1:34159)
0927 11:11:58.681226 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:58.681228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:58.681410 (+ 182us) server_negotiation.cc:407] Connection header received
0927 11:11:58.681440 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:58.681441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:58.681445 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:58.681507 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:58.682045 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:58.682047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:58.682047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:11:58.682051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:58.682064 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:11:58.682068 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:58.682133 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:58.682180 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 11:11:58.682192 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:11:58.697901 6112 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:58.698066 6112 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:58.704186 6112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8b268b607c04ed8bef001b917c5b66c: Generated new TSK 0
I20260927 11:11:58.704357 6112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:58.710196 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6037
2026-09-27T11:11:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:11:58 dist-test-slave-6j0v krb5kdc[6136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:11:58 dist-test-slave-6j0v krb5kdc[6136](info): set up 2 sockets
Sep 27 11:11:58 dist-test-slave-6j0v krb5kdc[6136](info): commencing operation
krb5kdc: starting...
W20260927 11:11:59.723021 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:11:59 dist-test-slave-6j0v krb5kdc[6136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:11:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:11:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:11:59.833863 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:11:59.910468 6149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:11:59.910691 6149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:11:59.910727 6149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:11:59.911820 6149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:11:59.911890 6149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:11:59.911914 6149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:11:59.911936 6149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:11:59.911963 6149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:11:59.913378 6149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:11:59.913650 6149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:11:59.913858 6149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:11:59.918515 6157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:59.918524 6154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:11:59.918519 6155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:11:59.918591 6149 server_base.cc:1034] running on GCE node
I20260927 11:11:59.918921 6149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:11:59.919239 6149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:11:59.920389 6149 hybrid_clock.cc:648] HybridClock initialized: now 1790507519920363 us; error 39 us; skew 500 ppm
Sep 27 11:11:59 dist-test-slave-6j0v krb5kdc[6136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507519, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:11:59.922884 6149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:11:59.923357 6149 webserver.cc:460] Webserver started at http://127.0.0.1:35171/ using document root <none> and password file <none>
I20260927 11:11:59.923571 6149 fs_manager.cc:359] Metadata directory not provided
I20260927 11:11:59.923636 6149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:11:59.923789 6149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:11:59.924798 6149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "019e97cae3484e74b9bf91100faaa314"
format_stamp: "Formatted at 2026-09-27 11:11:59 on dist-test-slave-6j0v"
server_key: "b959ad49348d66bdb825ba454f411569"
server_key_iv: "9abf51e6b27ae01e9f5f7b77d735e2c2"
server_key_version: "encryptionkey@0"
I20260927 11:11:59.925148 6149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "019e97cae3484e74b9bf91100faaa314"
format_stamp: "Formatted at 2026-09-27 11:11:59 on dist-test-slave-6j0v"
server_key: "b959ad49348d66bdb825ba454f411569"
server_key_iv: "9abf51e6b27ae01e9f5f7b77d735e2c2"
server_key_version: "encryptionkey@0"
I20260927 11:11:59.926438 6149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:11:59.927225 6164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:11:59.927443 6149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:11:59.927541 6149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "019e97cae3484e74b9bf91100faaa314"
format_stamp: "Formatted at 2026-09-27 11:11:59 on dist-test-slave-6j0v"
server_key: "b959ad49348d66bdb825ba454f411569"
server_key_iv: "9abf51e6b27ae01e9f5f7b77d735e2c2"
server_key_version: "encryptionkey@0"
I20260927 11:11:59.927626 6149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:11:59.946209 6149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:11:59.948065 6149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:11:59.948222 6149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:11:59.952092 6149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42743
I20260927 11:11:59.952123 6216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42743 every 8 connection(s)
I20260927 11:11:59.952481 6149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:11:59.953258 6217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:11:59.955519 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314: Bootstrap starting.
I20260927 11:11:59.956014 6217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314: Neither blocks nor log segments found. Creating new log.
I20260927 11:11:59.956255 6217 log.cc:824] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314: Log is configured to *not* fsync() on all Append() calls
I20260927 11:11:59.956848 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314: No bootstrap required, opened a new log
I20260927 11:11:59.957726 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6149
I20260927 11:11:59.957850 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:11:59.958039 17791 external_mini_cluster.cc:1428] Setting key 937387631ea74c97920f906f656b3f43
I20260927 11:11:59.959331 6217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019e97cae3484e74b9bf91100faaa314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260927 11:11:59.959487 6217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:11:59.959522 6217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e97cae3484e74b9bf91100faaa314, State: Initialized, Role: FOLLOWER
I20260927 11:11:59.959678 6217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019e97cae3484e74b9bf91100faaa314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260927 11:11:59.959738 6217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:11:59.959758 6217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:11:59.959816 6217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:11:59.960395 6217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019e97cae3484e74b9bf91100faaa314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260927 11:11:59.960500 6217 leader_election.cc:302] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 019e97cae3484e74b9bf91100faaa314; no voters:
I20260927 11:11:59.960659 6217 leader_election.cc:288] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:11:59.960739 6222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:11:59.960894 6217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:11:59.960903 6222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [term 1 LEADER]: Becoming Leader. State: Replica: 019e97cae3484e74b9bf91100faaa314, State: Running, Role: LEADER
I20260927 11:11:59.961061 6222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019e97cae3484e74b9bf91100faaa314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260927 11:11:59.961447 6224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 019e97cae3484e74b9bf91100faaa314. Latest consensus state: current_term: 1 leader_uuid: "019e97cae3484e74b9bf91100faaa314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019e97cae3484e74b9bf91100faaa314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } } }
I20260927 11:11:59.961534 6224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:59.961726 6223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "019e97cae3484e74b9bf91100faaa314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019e97cae3484e74b9bf91100faaa314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } } }
I20260927 11:11:59.962023 6223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314 [sys.catalog]: This master's current role is: LEADER
I20260927 11:11:59.962193 6226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:11:59.962647 6226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 11:11:59 dist-test-slave-6j0v krb5kdc[6136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 11:11:59.967597 6226 catalog_manager.cc:1347] Generated new cluster ID: 218055c1ea1b48b6a39fc511978f2c2b
I20260927 11:11:59.967650 6226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:11:59.969996 6221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:59.958559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:36645 (local address 127.0.0.1:42743)
0927 11:11:59.958770 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:59.958780 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:59.958815 (+ 35us) server_negotiation.cc:407] Connection header received
0927 11:11:59.959271 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:59.959280 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:59.959363 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:59.959496 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:59.960239 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:59.961593 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:59.962741 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:59.963315 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:59.963411 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:59.966037 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:59.966041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:59.966041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:59.966071 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:59.967916 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:59.968379 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:59.968382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:59.968383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:59.968501 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:59.968741 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:59.968742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:59.968742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:59.969030 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:59.969185 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:59.969679 (+ 494us) server_negotiation.cc:299] Negotiation successful
0927 11:11:59.969781 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260927 11:11:59.970908 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:11:59.990720 6221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:11:59.983374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:45163 (local address 127.0.0.1:42743)
0927 11:11:59.983451 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 11:11:59.983453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:11:59.983517 (+ 64us) server_negotiation.cc:407] Connection header received
0927 11:11:59.983575 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:11:59.983576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:11:59.983587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:11:59.983646 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:11:59.984281 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:59.984983 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:59.986017 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:11:59.986325 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:11:59.986382 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:11:59.987842 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:11:59.987844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:11:59.987844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:11:59.987850 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:11:59.989404 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:59.989812 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:59.989813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:59.989813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:59.989885 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:11:59.990119 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:11:59.990120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:11:59.990120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:11:59.990257 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:11:59.990358 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 11:11:59.990549 (+ 191us) server_negotiation.cc:299] Negotiation successful
0927 11:11:59.990618 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 11:11:59.990784 6226 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:11:59.991684 6226 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:11:59.995224 6226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 019e97cae3484e74b9bf91100faaa314: Generated new TSK 0
I20260927 11:11:59.995409 6226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:11:59.998329 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6149
W20260927 11:12:00.002250 6134 connection.cc:535] client connection to 127.0.0.1:42743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42743): Connection reset by peer (error 104)
2026-09-27T11:11:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:00 dist-test-slave-6j0v krb5kdc[6248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:00 dist-test-slave-6j0v krb5kdc[6248](info): set up 2 sockets
Sep 27 11:12:00 dist-test-slave-6j0v krb5kdc[6248](info): commencing operation
krb5kdc: starting...
W20260927 11:12:01.015405 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:01 dist-test-slave-6j0v krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:12:01.128049 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:01.205458 6261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:01.205649 6261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:01.205672 6261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:01.206770 6261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:01.206805 6261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:01.206820 6261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:01.206832 6261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:01.206843 6261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:01.208287 6261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:01.208460 6261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:01.208629 6261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:01.213338 6266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:01.213429 6261 server_base.cc:1034] running on GCE node
W20260927 11:12:01.213347 6267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:01.213352 6269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:01.213766 6261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:01.214030 6261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:01.215133 6261 hybrid_clock.cc:648] HybridClock initialized: now 1790507521215124 us; error 33 us; skew 500 ppm
Sep 27 11:12:01 dist-test-slave-6j0v krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507521, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:01.217407 6261 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:12:01.217784 6261 webserver.cc:460] Webserver started at http://127.0.0.1:36615/ using document root <none> and password file <none>
I20260927 11:12:01.217979 6261 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:01.218039 6261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:01.218190 6261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:01.219256 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "e3223211cd094509801a0f91a931009d"
format_stamp: "Formatted at 2026-09-27 11:12:01 on dist-test-slave-6j0v"
server_key: "cc3e4e635ce345d6cf77f2ec17138aaf"
server_key_iv: "c298b6e4f4349910570738fca325cd09"
server_key_version: "encryptionkey@0"
I20260927 11:12:01.219576 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "e3223211cd094509801a0f91a931009d"
format_stamp: "Formatted at 2026-09-27 11:12:01 on dist-test-slave-6j0v"
server_key: "cc3e4e635ce345d6cf77f2ec17138aaf"
server_key_iv: "c298b6e4f4349910570738fca325cd09"
server_key_version: "encryptionkey@0"
I20260927 11:12:01.220789 6261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:01.221552 6276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:01.221758 6261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:01.221846 6261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "e3223211cd094509801a0f91a931009d"
format_stamp: "Formatted at 2026-09-27 11:12:01 on dist-test-slave-6j0v"
server_key: "cc3e4e635ce345d6cf77f2ec17138aaf"
server_key_iv: "c298b6e4f4349910570738fca325cd09"
server_key_version: "encryptionkey@0"
I20260927 11:12:01.221920 6261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:01.232375 6261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:01.234028 6261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:01.234170 6261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:01.237852 6261 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39889
I20260927 11:12:01.237900 6328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39889 every 8 connection(s)
I20260927 11:12:01.238211 6261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:01.239012 6329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:01.241106 6329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d: Bootstrap starting.
I20260927 11:12:01.241632 6329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:01.241804 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6261
I20260927 11:12:01.241914 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:01.241935 6329 log.cc:824] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:01.242113 17791 external_mini_cluster.cc:1428] Setting key e614644976c96ffce55dd8c63d39a085
I20260927 11:12:01.242681 6329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d: No bootstrap required, opened a new log
I20260927 11:12:01.244640 6329 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3223211cd094509801a0f91a931009d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260927 11:12:01.244793 6329 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:01.244840 6329 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3223211cd094509801a0f91a931009d, State: Initialized, Role: FOLLOWER
I20260927 11:12:01.244966 6329 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3223211cd094509801a0f91a931009d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260927 11:12:01.245069 6329 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:01.245112 6329 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:01.245162 6329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:01.245676 6329 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3223211cd094509801a0f91a931009d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260927 11:12:01.245800 6329 leader_election.cc:302] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3223211cd094509801a0f91a931009d; no voters:
I20260927 11:12:01.245991 6329 leader_election.cc:288] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:01.246013 6334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:01.246126 6334 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [term 1 LEADER]: Becoming Leader. State: Replica: e3223211cd094509801a0f91a931009d, State: Running, Role: LEADER
I20260927 11:12:01.246254 6329 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:01.246277 6334 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3223211cd094509801a0f91a931009d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260927 11:12:01.246714 6336 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3223211cd094509801a0f91a931009d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3223211cd094509801a0f91a931009d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } } }
I20260927 11:12:01.246734 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3223211cd094509801a0f91a931009d. Latest consensus state: current_term: 1 leader_uuid: "e3223211cd094509801a0f91a931009d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3223211cd094509801a0f91a931009d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } } }
I20260927 11:12:01.246889 6336 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:01.247155 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:01.247176 6343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:01.247740 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:01.242555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:33811 (local address 127.0.0.1:39889)
0927 11:12:01.242715 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:01.242724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:01.242765 (+ 41us) server_negotiation.cc:407] Connection header received
0927 11:12:01.243187 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:01.243192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:01.243235 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:01.243365 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:01.243563 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:01.244415 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:01.245404 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:01.245849 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:01.245972 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:01.246864 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:01.246867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:01.246868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:01.247102 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:01.247144 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:01.247159 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:01.247243 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:01.247514 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 11:12:01.247542 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260927 11:12:01.247773 6343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:01.248673 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:01.249496 6343 catalog_manager.cc:1347] Generated new cluster ID: e8c8243fac204e799c6adb073e39eef3
I20260927 11:12:01.249537 6343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:01.254402 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:01.250142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:52359 (local address 127.0.0.1:39889)
0927 11:12:01.251144 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:01.251145 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:01.251152 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:12:01.251169 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:01.251170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:01.251178 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:01.251224 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:01.251398 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:01.252090 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:01.253128 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:01.253419 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:01.253475 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:01.254165 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:01.254166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:01.254167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:01.254171 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:01.254184 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:01.254188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:01.254240 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:01.254324 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 11:12:01.254332 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":984}
I20260927 11:12:01.281695 6343 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:01.282466 6343 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:01.286440 6343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3223211cd094509801a0f91a931009d: Generated new TSK 0
I20260927 11:12:01.286635 6343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:01.293742 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6261
W20260927 11:12:01.298182 6246 connection.cc:535] client connection to 127.0.0.1:39889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39889): Connection reset by peer (error 104)
2026-09-27T11:12:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:01 dist-test-slave-6j0v krb5kdc[6360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:01 dist-test-slave-6j0v krb5kdc[6360](info): set up 2 sockets
Sep 27 11:12:01 dist-test-slave-6j0v krb5kdc[6360](info): commencing operation
krb5kdc: starting...
W20260927 11:12:02.287591 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:02 dist-test-slave-6j0v krb5kdc[6360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:12:02.399569 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:02.477566 6373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:02.477798 6373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:02.477836 6373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:02.479003 6373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:02.479071 6373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:02.479120 6373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:02.479161 6373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:02.479200 6373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:02.480607 6373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:02.480870 6373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:02.481078 6373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:02.485721 6378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:02.485912 6373 server_base.cc:1034] running on GCE node
W20260927 11:12:02.485843 6379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:02.485715 6381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:02.486330 6373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:02.486588 6373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:02.487758 6373 hybrid_clock.cc:648] HybridClock initialized: now 1790507522487751 us; error 35 us; skew 500 ppm
Sep 27 11:12:02 dist-test-slave-6j0v krb5kdc[6360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507522, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:02.490330 6373 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:12:02.490742 6373 webserver.cc:460] Webserver started at http://127.0.0.1:38403/ using document root <none> and password file <none>
I20260927 11:12:02.490944 6373 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:02.490988 6373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:02.491163 6373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:02.492219 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "0e0c7738c9f74723a73742d518fef066"
format_stamp: "Formatted at 2026-09-27 11:12:02 on dist-test-slave-6j0v"
server_key: "9bb78a2be9fe6e98aed07a6bb3933972"
server_key_iv: "9c18bbe00ad9f184f41c52a65b99976d"
server_key_version: "encryptionkey@0"
I20260927 11:12:02.492547 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "0e0c7738c9f74723a73742d518fef066"
format_stamp: "Formatted at 2026-09-27 11:12:02 on dist-test-slave-6j0v"
server_key: "9bb78a2be9fe6e98aed07a6bb3933972"
server_key_iv: "9c18bbe00ad9f184f41c52a65b99976d"
server_key_version: "encryptionkey@0"
I20260927 11:12:02.493945 6373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:12:02.494692 6388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:02.494869 6373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:02.494946 6373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "0e0c7738c9f74723a73742d518fef066"
format_stamp: "Formatted at 2026-09-27 11:12:02 on dist-test-slave-6j0v"
server_key: "9bb78a2be9fe6e98aed07a6bb3933972"
server_key_iv: "9c18bbe00ad9f184f41c52a65b99976d"
server_key_version: "encryptionkey@0"
I20260927 11:12:02.494998 6373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:02.499169 6373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:02.501570 6373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:02.501715 6373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:02.506862 6373 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44313
I20260927 11:12:02.506901 6440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44313 every 8 connection(s)
I20260927 11:12:02.507325 6373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:02.508438 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:02.511852 6441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066: Bootstrap starting.
I20260927 11:12:02.512530 6441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:02.512863 6441 log.cc:824] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:02.513541 6441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066: No bootstrap required, opened a new log
I20260927 11:12:02.513738 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6373
I20260927 11:12:02.513856 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:02.514074 17791 external_mini_cluster.cc:1428] Setting key b19da001c3d444b284fa504199b91358
I20260927 11:12:02.516044 6441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0c7738c9f74723a73742d518fef066" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44313 } }
I20260927 11:12:02.516204 6441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:02.516283 6441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0c7738c9f74723a73742d518fef066, State: Initialized, Role: FOLLOWER
I20260927 11:12:02.516412 6441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0c7738c9f74723a73742d518fef066" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44313 } }
I20260927 11:12:02.516516 6441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:02.516559 6441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:02.516601 6441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:02.517405 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:02.514561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:53227 (local address 127.0.0.1:44313)
0927 11:12:02.514704 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:02.514713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:02.515469 (+ 756us) server_negotiation.cc:407] Connection header received
0927 11:12:02.516001 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:02.516009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:02.516045 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:02.516191 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:02.516957 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:02.516960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:02.516961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:02.516965 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:02.517012 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:02.517023 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:02.517103 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:02.517173 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 11:12:02.517218 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260927 11:12:02.517511 6441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0c7738c9f74723a73742d518fef066" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44313 } }
I20260927 11:12:02.517634 6441 leader_election.cc:302] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e0c7738c9f74723a73742d518fef066; no voters:
I20260927 11:12:02.517819 6441 leader_election.cc:288] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:02.517853 6446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:02.518008 6446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [term 1 LEADER]: Becoming Leader. State: Replica: 0e0c7738c9f74723a73742d518fef066, State: Running, Role: LEADER
I20260927 11:12:02.518051 6441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:02.518121 6446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0c7738c9f74723a73742d518fef066" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44313 } }
I20260927 11:12:02.518258 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:02.518461 6446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e0c7738c9f74723a73742d518fef066. Latest consensus state: current_term: 1 leader_uuid: "0e0c7738c9f74723a73742d518fef066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0c7738c9f74723a73742d518fef066" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44313 } } }
I20260927 11:12:02.518549 6446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:02.518472 6448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e0c7738c9f74723a73742d518fef066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0c7738c9f74723a73742d518fef066" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44313 } } }
I20260927 11:12:02.518682 6448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066 [sys.catalog]: This master's current role is: LEADER
W20260927 11:12:02.520156 6464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:02.520221 6464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:02.520282 6452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:02.520447 6452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:02.520901 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:02.519766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:56299 (local address 127.0.0.1:44313)
0927 11:12:02.519795 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:02.519797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:02.519933 (+ 136us) server_negotiation.cc:407] Connection header received
0927 11:12:02.520013 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:02.520014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:02.520019 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:02.520078 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:02.520696 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:02.520697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:02.520697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:02.520700 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:02.520713 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:02.520717 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:02.520753 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:02.520826 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 11:12:02.520836 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260927 11:12:02.521919 6452 catalog_manager.cc:1347] Generated new cluster ID: a6906f60d2ca4a4d8bf199ce7d9ec4ff
I20260927 11:12:02.521967 6452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:02.529383 6452 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:02.529561 6452 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:02.531934 6452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e0c7738c9f74723a73742d518fef066: Generated new TSK 0
I20260927 11:12:02.532074 6452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:02.539711 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6373
2026-09-27T11:12:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:02 dist-test-slave-6j0v krb5kdc[6472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:02 dist-test-slave-6j0v krb5kdc[6472](info): set up 2 sockets
Sep 27 11:12:02 dist-test-slave-6j0v krb5kdc[6472](info): commencing operation
krb5kdc: starting...
W20260927 11:12:03.539032 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:03 dist-test-slave-6j0v krb5kdc[6472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:12:03.650183 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:03.732369 6485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:03.732579 6485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:03.732600 6485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:03.733789 6485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:03.733834 6485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:03.733847 6485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:03.733860 6485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:03.733914 6485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:03.735441 6485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:03.735700 6485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:03.735945 6485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:03.740612 6490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:03.740640 6493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:03.740612 6491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:03.740921 6485 server_base.cc:1034] running on GCE node
I20260927 11:12:03.741165 6485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:03.741492 6485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:03.742661 6485 hybrid_clock.cc:648] HybridClock initialized: now 1790507523742656 us; error 37 us; skew 500 ppm
Sep 27 11:12:03 dist-test-slave-6j0v krb5kdc[6472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507523, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:03.748997 6485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:12:03.749452 6485 webserver.cc:460] Webserver started at http://127.0.0.1:46063/ using document root <none> and password file <none>
I20260927 11:12:03.749662 6485 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:03.749702 6485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:03.749788 6485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:03.750814 6485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "042568d9d6104fe7bf450e4517b674e2"
format_stamp: "Formatted at 2026-09-27 11:12:03 on dist-test-slave-6j0v"
server_key: "0d25fa51022e54530b2accacda33baef"
server_key_iv: "dd6374db226ddc53c899dff8aaf1947f"
server_key_version: "encryptionkey@0"
I20260927 11:12:03.751252 6485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "042568d9d6104fe7bf450e4517b674e2"
format_stamp: "Formatted at 2026-09-27 11:12:03 on dist-test-slave-6j0v"
server_key: "0d25fa51022e54530b2accacda33baef"
server_key_iv: "dd6374db226ddc53c899dff8aaf1947f"
server_key_version: "encryptionkey@0"
I20260927 11:12:03.752537 6485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:03.753216 6500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:03.753394 6485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:12:03.753477 6485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "042568d9d6104fe7bf450e4517b674e2"
format_stamp: "Formatted at 2026-09-27 11:12:03 on dist-test-slave-6j0v"
server_key: "0d25fa51022e54530b2accacda33baef"
server_key_iv: "dd6374db226ddc53c899dff8aaf1947f"
server_key_version: "encryptionkey@0"
I20260927 11:12:03.753528 6485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:03.790499 6485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:03.792646 6485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:03.792755 6485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:03.796767 6485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39267
I20260927 11:12:03.796828 6552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39267 every 8 connection(s)
I20260927 11:12:03.797156 6485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:03.798012 6553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:03.800267 6553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2: Bootstrap starting.
I20260927 11:12:03.800866 6553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:03.801148 6553 log.cc:824] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:03.801899 6553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2: No bootstrap required, opened a new log
I20260927 11:12:03.803571 6553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042568d9d6104fe7bf450e4517b674e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260927 11:12:03.803714 6553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:03.803736 6553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042568d9d6104fe7bf450e4517b674e2, State: Initialized, Role: FOLLOWER
I20260927 11:12:03.803929 6553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042568d9d6104fe7bf450e4517b674e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260927 11:12:03.804015 6553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:03.804057 6553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:03.804108 6553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:03.804659 6553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042568d9d6104fe7bf450e4517b674e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260927 11:12:03.804745 6553 leader_election.cc:302] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 042568d9d6104fe7bf450e4517b674e2; no voters:
I20260927 11:12:03.804922 6553 leader_election.cc:288] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:03.805007 6556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:03.805157 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6485
I20260927 11:12:03.805184 6553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:03.805259 6556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [term 1 LEADER]: Becoming Leader. State: Replica: 042568d9d6104fe7bf450e4517b674e2, State: Running, Role: LEADER
I20260927 11:12:03.805281 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:03.805425 6556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042568d9d6104fe7bf450e4517b674e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } }
I20260927 11:12:03.805483 17791 external_mini_cluster.cc:1428] Setting key 270fd07b28047e792100e686f01990c5
I20260927 11:12:03.805896 6557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "042568d9d6104fe7bf450e4517b674e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042568d9d6104fe7bf450e4517b674e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } } }
I20260927 11:12:03.805981 6557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:03.806139 6558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 042568d9d6104fe7bf450e4517b674e2. Latest consensus state: current_term: 1 leader_uuid: "042568d9d6104fe7bf450e4517b674e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "042568d9d6104fe7bf450e4517b674e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39267 } } }
I20260927 11:12:03.806176 6567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:03.806224 6558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:03.806705 6567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:03.808445 6567 catalog_manager.cc:1347] Generated new cluster ID: effbe8d93ef74dd0b5f4a173101df622
I20260927 11:12:03.808501 6567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:12:03 dist-test-slave-6j0v krb5kdc[6472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 11:12:03.816556 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:03.805952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:48901 (local address 127.0.0.1:39267)
0927 11:12:03.806102 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:03.806113 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:03.806445 (+ 332us) server_negotiation.cc:407] Connection header received
0927 11:12:03.806675 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:03.806682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:03.806804 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:03.806925 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:03.807994 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:03.809040 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:03.809808 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:03.810230 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:03.810346 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:03.812770 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:03.812774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:03.812775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:03.812787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:03.814935 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:03.815365 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:03.815368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:03.815368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:03.815471 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:03.815723 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:03.815724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:03.815724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:03.815981 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:03.816187 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:03.816282 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 11:12:03.816400 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260927 11:12:03.817500 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:03.824959 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:03.818793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:57075 (local address 127.0.0.1:39267)
0927 11:12:03.818835 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:03.818836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:03.819272 (+ 436us) server_negotiation.cc:407] Connection header received
0927 11:12:03.819334 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:03.819335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:03.819344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:03.819398 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:03.819920 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:03.820398 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:03.821040 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:03.821255 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:03.821337 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:03.822455 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:03.822457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:03.822457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:03.822464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:03.823874 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:03.824217 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:03.824218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:03.824219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:03.824267 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:03.824473 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:03.824474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:03.824474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:03.824578 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:03.824685 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:03.824806 (+ 121us) server_negotiation.cc:299] Negotiation successful
0927 11:12:03.824865 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:12:03.828231 6567 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:03.828830 6567 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:03.835651 6567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 042568d9d6104fe7bf450e4517b674e2: Generated new TSK 0
I20260927 11:12:03.835853 6567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:03.843417 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6485
W20260927 11:12:03.847780 6470 connection.cc:535] client connection to 127.0.0.1:39267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39267): Connection reset by peer (error 104)
2026-09-27T11:12:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:03 dist-test-slave-6j0v krb5kdc[6584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:03 dist-test-slave-6j0v krb5kdc[6584](info): set up 2 sockets
Sep 27 11:12:03 dist-test-slave-6j0v krb5kdc[6584](info): commencing operation
krb5kdc: starting...
W20260927 11:12:04.871625 17791 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:04 dist-test-slave-6j0v krb5kdc[6584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:12:04.985212 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41061
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:05.062768 6597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:05.062999 6597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:05.063036 6597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:05.064188 6597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:05.064256 6597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:05.064280 6597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:05.064299 6597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:05.064328 6597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:05.065694 6597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41061
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:05.065970 6597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:05.066195 6597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:05.070927 6603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:05.070926 6602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:05.071022 6597 server_base.cc:1034] running on GCE node
W20260927 11:12:05.070932 6605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:05.071398 6597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:05.071674 6597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:05.072839 6597 hybrid_clock.cc:648] HybridClock initialized: now 1790507525072833 us; error 47 us; skew 500 ppm
Sep 27 11:12:05 dist-test-slave-6j0v krb5kdc[6584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:05.075191 6597 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:12:05.075541 6597 webserver.cc:460] Webserver started at http://127.0.0.1:36167/ using document root <none> and password file <none>
I20260927 11:12:05.075708 6597 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:05.075745 6597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:05.075826 6597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:05.076786 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "ebd4c538e26e468c986fc7b79a4a87ae"
format_stamp: "Formatted at 2026-09-27 11:12:05 on dist-test-slave-6j0v"
server_key: "9a809e76049a35703d87f9b7baece749"
server_key_iv: "7fdaa52490aa4fbcb00c67fce7de9426"
server_key_version: "encryptionkey@0"
I20260927 11:12:05.077069 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "ebd4c538e26e468c986fc7b79a4a87ae"
format_stamp: "Formatted at 2026-09-27 11:12:05 on dist-test-slave-6j0v"
server_key: "9a809e76049a35703d87f9b7baece749"
server_key_iv: "7fdaa52490aa4fbcb00c67fce7de9426"
server_key_version: "encryptionkey@0"
I20260927 11:12:05.078292 6597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:12:05.078969 6612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:05.079131 6597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:05.079211 6597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "ebd4c538e26e468c986fc7b79a4a87ae"
format_stamp: "Formatted at 2026-09-27 11:12:05 on dist-test-slave-6j0v"
server_key: "9a809e76049a35703d87f9b7baece749"
server_key_iv: "7fdaa52490aa4fbcb00c67fce7de9426"
server_key_version: "encryptionkey@0"
I20260927 11:12:05.079268 6597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:05.104300 6597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:05.107125 6597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:05.107303 6597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:05.111236 6597 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41061
I20260927 11:12:05.111308 6664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41061 every 8 connection(s)
I20260927 11:12:05.111651 6597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:05.112434 6665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:05.114543 6665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae: Bootstrap starting.
I20260927 11:12:05.115061 6665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:05.115379 6665 log.cc:824] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:05.116016 6665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae: No bootstrap required, opened a new log
I20260927 11:12:05.117610 6665 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41061 } }
I20260927 11:12:05.117807 6665 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:05.117852 6665 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd4c538e26e468c986fc7b79a4a87ae, State: Initialized, Role: FOLLOWER
I20260927 11:12:05.118017 6665 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41061 } }
I20260927 11:12:05.118113 6665 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:05.118161 6665 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:05.118211 6665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:05.118674 6665 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41061 } }
I20260927 11:12:05.118778 6665 leader_election.cc:302] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebd4c538e26e468c986fc7b79a4a87ae; no voters:
I20260927 11:12:05.118952 6665 leader_election.cc:288] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:05.118970 6668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:05.119138 6668 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [term 1 LEADER]: Becoming Leader. State: Replica: ebd4c538e26e468c986fc7b79a4a87ae, State: Running, Role: LEADER
I20260927 11:12:05.119212 6665 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:05.119223 6668 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41061 } }
I20260927 11:12:05.119555 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6597
I20260927 11:12:05.119587 6669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41061 } } }
I20260927 11:12:05.119669 6669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:05.119692 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:05.119895 17791 external_mini_cluster.cc:1428] Setting key b0aab45c2eb01f5a17add39d90c6cd63
I20260927 11:12:05.119880 6675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:05.119865 6670 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebd4c538e26e468c986fc7b79a4a87ae. Latest consensus state: current_term: 1 leader_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd4c538e26e468c986fc7b79a4a87ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41061 } } }
I20260927 11:12:05.120059 6670 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:05.120473 6675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:05.122835 6675 catalog_manager.cc:1347] Generated new cluster ID: e43e4a9e38f2499a8c5d9ca1506a9139
I20260927 11:12:05.122888 6675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:05.125950 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:05.120365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:45059 (local address 127.0.0.1:41061)
0927 11:12:05.121294 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:05.121303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:05.121468 (+ 165us) server_negotiation.cc:407] Connection header received
0927 11:12:05.121588 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:05.121595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:05.121665 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:05.121811 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:05.122094 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:05.123283 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:05.124071 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:05.124509 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:05.124640 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:05.125339 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:05.125343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:05.125343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:05.125511 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:05.125540 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:05.125550 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:05.125637 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:05.125719 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 11:12:05.125747 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":97,"threads_started":1}
I20260927 11:12:05.126861 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:05.134239 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:05.131431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:38231 (local address 127.0.0.1:41061)
0927 11:12:05.131467 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:05.131469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:05.131601 (+ 132us) server_negotiation.cc:407] Connection header received
0927 11:12:05.131682 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:05.131683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:05.131695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:05.131748 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:05.131876 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:05.132341 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:05.133067 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:05.133291 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:05.133371 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:05.134002 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:05.134004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:05.134004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:05.134008 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:05.134021 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:05.134025 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:05.134075 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:05.134138 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 11:12:05.134150 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:12:05.158041 6675 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:05.158560 6675 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:05.161180 6675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebd4c538e26e468c986fc7b79a4a87ae: Generated new TSK 0
I20260927 11:12:05.161371 6675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:05.170432 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6597
W20260927 11:12:05.174994 6582 connection.cc:535] client connection to 127.0.0.1:41061 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41061): Connection reset by peer (error 104)
2026-09-27T11:12:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:05 dist-test-slave-6j0v krb5kdc[6696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:05 dist-test-slave-6j0v krb5kdc[6696](info): set up 2 sockets
Sep 27 11:12:05 dist-test-slave-6j0v krb5kdc[6696](info): commencing operation
krb5kdc: starting...
W20260927 11:12:06.167040 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:06 dist-test-slave-6j0v krb5kdc[6696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 11:12:06.282330 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:06.360157 6709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:06.360560 6709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:06.360649 6709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:06.361756 6709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:06.361824 6709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:06.361848 6709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:06.361879 6709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:06.361908 6709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:06.363399 6709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:06.363659 6709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:06.363907 6709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:06.370117 6714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:06.370144 6715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:06.370544 6717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:06.371058 6709 server_base.cc:1034] running on GCE node
I20260927 11:12:06.371342 6709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:06.371678 6709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:06.372844 6709 hybrid_clock.cc:648] HybridClock initialized: now 1790507526372820 us; error 38 us; skew 500 ppm
Sep 27 11:12:06 dist-test-slave-6j0v krb5kdc[6696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507526, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:06.376235 6709 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 11:12:06.376720 6709 webserver.cc:460] Webserver started at http://127.0.0.1:41377/ using document root <none> and password file <none>
I20260927 11:12:06.377089 6709 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:06.377151 6709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:06.377285 6709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:06.378870 6709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "ec4f2b0de11b4c80b53f75c7ece40d47"
format_stamp: "Formatted at 2026-09-27 11:12:06 on dist-test-slave-6j0v"
server_key: "c3b09b89d113176b0c381057b3c45adf"
server_key_iv: "9e9e2e4e0276ce9993e4eb33b842a34b"
server_key_version: "encryptionkey@0"
I20260927 11:12:06.379308 6709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "ec4f2b0de11b4c80b53f75c7ece40d47"
format_stamp: "Formatted at 2026-09-27 11:12:06 on dist-test-slave-6j0v"
server_key: "c3b09b89d113176b0c381057b3c45adf"
server_key_iv: "9e9e2e4e0276ce9993e4eb33b842a34b"
server_key_version: "encryptionkey@0"
I20260927 11:12:06.381121 6709 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260927 11:12:06.381980 6724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:06.382160 6709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:06.382246 6709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "ec4f2b0de11b4c80b53f75c7ece40d47"
format_stamp: "Formatted at 2026-09-27 11:12:06 on dist-test-slave-6j0v"
server_key: "c3b09b89d113176b0c381057b3c45adf"
server_key_iv: "9e9e2e4e0276ce9993e4eb33b842a34b"
server_key_version: "encryptionkey@0"
I20260927 11:12:06.382319 6709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:06.387281 6709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:06.389842 6709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:06.389976 6709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:06.394733 6709 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42865
I20260927 11:12:06.394761 6776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42865 every 8 connection(s)
I20260927 11:12:06.395237 6709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:06.396082 6777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:06.396641 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6709
I20260927 11:12:06.396792 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:06.397022 17791 external_mini_cluster.cc:1428] Setting key e99ab1a3fb393d4126123a7d99ee70f5
I20260927 11:12:06.398931 6777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47: Bootstrap starting.
I20260927 11:12:06.399646 6777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:06.399974 6777 log.cc:824] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:06.400244 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:06.397688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:55351 (local address 127.0.0.1:42865)
0927 11:12:06.397891 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:06.397899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:06.398226 (+ 327us) server_negotiation.cc:407] Connection header received
0927 11:12:06.398723 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:06.398731 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:06.398765 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:06.398910 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:06.399557 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:06.399560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:06.399560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:06.399565 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:06.399606 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:06.399620 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:06.399909 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:06.399942 (+ 33us) server_negotiation.cc:299] Negotiation successful
0927 11:12:06.399984 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260927 11:12:06.400717 6777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47: No bootstrap required, opened a new log
I20260927 11:12:06.401161 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:06.403285 6777 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42865 } }
I20260927 11:12:06.403438 6777 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:06.403483 6777 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec4f2b0de11b4c80b53f75c7ece40d47, State: Initialized, Role: FOLLOWER
I20260927 11:12:06.403586 6777 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42865 } }
I20260927 11:12:06.403689 6777 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:06.403725 6777 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:06.403769 6777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:06.403993 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:06.402646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:33159 (local address 127.0.0.1:42865)
0927 11:12:06.402685 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:06.402687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:06.402791 (+ 104us) server_negotiation.cc:407] Connection header received
0927 11:12:06.402851 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:06.402852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:06.402856 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:06.402923 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:06.403594 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:06.403595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:06.403595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:06.403599 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:06.403611 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:06.403614 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:06.403698 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:06.403930 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 11:12:06.403943 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:12:06.404453 6777 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42865 } }
I20260927 11:12:06.404559 6777 leader_election.cc:302] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec4f2b0de11b4c80b53f75c7ece40d47; no voters:
I20260927 11:12:06.404707 6777 leader_election.cc:288] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:06.404811 6787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:06.404945 6777 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:06.405059 6787 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [term 1 LEADER]: Becoming Leader. State: Replica: ec4f2b0de11b4c80b53f75c7ece40d47, State: Running, Role: LEADER
I20260927 11:12:06.405223 6787 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42865 } }
I20260927 11:12:06.405663 6787 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec4f2b0de11b4c80b53f75c7ece40d47. Latest consensus state: current_term: 1 leader_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42865 } } }
I20260927 11:12:06.405694 6788 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f2b0de11b4c80b53f75c7ece40d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42865 } } }
I20260927 11:12:06.405809 6787 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:06.405822 6788 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:06.406165 6799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:06.406656 6799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:06.408200 6799 catalog_manager.cc:1347] Generated new cluster ID: 0630ebc789a1428bb3b76771fa784b0e
I20260927 11:12:06.408250 6799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:06.432693 6799 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:06.432863 6799 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:06.440692 6799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec4f2b0de11b4c80b53f75c7ece40d47: Generated new TSK 0
I20260927 11:12:06.440866 6799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:06.450042 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6709
2026-09-27T11:12:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:06 dist-test-slave-6j0v krb5kdc[6808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:06 dist-test-slave-6j0v krb5kdc[6808](info): set up 2 sockets
Sep 27 11:12:06 dist-test-slave-6j0v krb5kdc[6808](info): commencing operation
krb5kdc: starting...
W20260927 11:12:07.451061 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:07 dist-test-slave-6j0v krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:07.564254 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:38957
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:38957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:07.642071 6821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:07.642256 6821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:07.642277 6821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:07.643334 6821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:07.643366 6821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:07.643380 6821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:07.643405 6821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:07.643417 6821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:07.644835 6821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:38957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:38957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:07.645018 6821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:07.645186 6821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:07.649922 6829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:07.649955 6827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:07.650056 6821 server_base.cc:1034] running on GCE node
W20260927 11:12:07.649951 6826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:07.650367 6821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:07.650660 6821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:07.651825 6821 hybrid_clock.cc:648] HybridClock initialized: now 1790507527651801 us; error 41 us; skew 500 ppm
Sep 27 11:12:07 dist-test-slave-6j0v krb5kdc[6808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507527, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:07.654289 6821 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:07.654726 6821 webserver.cc:460] Webserver started at http://127.17.95.254:34059/ using document root <none> and password file <none>
I20260927 11:12:07.654929 6821 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:07.654991 6821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:07.655158 6821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:07.656111 6821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "21a80f819dc44891919c6cdaf81dafa1"
format_stamp: "Formatted at 2026-09-27 11:12:07 on dist-test-slave-6j0v"
server_key: "4c9746dfcc16a60ff77b742fe5a0b68c"
server_key_iv: "16edae44dd2c40c6008a73f6449b9468"
server_key_version: "encryptionkey@0"
I20260927 11:12:07.656468 6821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "21a80f819dc44891919c6cdaf81dafa1"
format_stamp: "Formatted at 2026-09-27 11:12:07 on dist-test-slave-6j0v"
server_key: "4c9746dfcc16a60ff77b742fe5a0b68c"
server_key_iv: "16edae44dd2c40c6008a73f6449b9468"
server_key_version: "encryptionkey@0"
I20260927 11:12:07.657764 6821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:12:07.658499 6836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:07.658710 6821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:07.658799 6821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "21a80f819dc44891919c6cdaf81dafa1"
format_stamp: "Formatted at 2026-09-27 11:12:07 on dist-test-slave-6j0v"
server_key: "4c9746dfcc16a60ff77b742fe5a0b68c"
server_key_iv: "16edae44dd2c40c6008a73f6449b9468"
server_key_version: "encryptionkey@0"
I20260927 11:12:07.658871 6821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:07.675580 6821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:07.677357 6821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:07.677505 6821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:07.681507 6821 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:38957
I20260927 11:12:07.681540 6888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:38957 every 8 connection(s)
I20260927 11:12:07.681854 6821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:07.682646 6889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:07.684868 6889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1: Bootstrap starting.
I20260927 11:12:07.685422 6889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:07.685689 6889 log.cc:824] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:07.686496 6889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1: No bootstrap required, opened a new log
I20260927 11:12:07.688169 6889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a80f819dc44891919c6cdaf81dafa1" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38957 } }
I20260927 11:12:07.688325 6889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:07.688365 6889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a80f819dc44891919c6cdaf81dafa1, State: Initialized, Role: FOLLOWER
I20260927 11:12:07.688491 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6821
I20260927 11:12:07.688519 6889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a80f819dc44891919c6cdaf81dafa1" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38957 } }
I20260927 11:12:07.688609 6889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:07.688624 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:07.688642 6889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:07.688696 6889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:07.688825 17791 external_mini_cluster.cc:1428] Setting key 66bd6cf5e63c8c25dd515e05cf8a9ca6
I20260927 11:12:07.689739 6889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a80f819dc44891919c6cdaf81dafa1" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38957 } }
I20260927 11:12:07.689850 6889 leader_election.cc:302] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a80f819dc44891919c6cdaf81dafa1; no voters:
I20260927 11:12:07.690006 6889 leader_election.cc:288] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:07.690030 6894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:07.690239 6894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [term 1 LEADER]: Becoming Leader. State: Replica: 21a80f819dc44891919c6cdaf81dafa1, State: Running, Role: LEADER
I20260927 11:12:07.690260 6889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:07.690336 6894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a80f819dc44891919c6cdaf81dafa1" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38957 } }
I20260927 11:12:07.690652 6894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a80f819dc44891919c6cdaf81dafa1. Latest consensus state: current_term: 1 leader_uuid: "21a80f819dc44891919c6cdaf81dafa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a80f819dc44891919c6cdaf81dafa1" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38957 } } }
I20260927 11:12:07.690721 6894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:07.690899 6900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:07.691376 6900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:07.692099 6895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a80f819dc44891919c6cdaf81dafa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a80f819dc44891919c6cdaf81dafa1" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 38957 } } }
I20260927 11:12:07.692198 6895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:07.692827 6900 catalog_manager.cc:1347] Generated new cluster ID: d3ee3a58c9164299a0932cac7418ad3b
I20260927 11:12:07.692880 6900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:12:07 dist-test-slave-6j0v krb5kdc[6808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:07.698686 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:07.689268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37587 (local address 127.17.95.254:38957)
0927 11:12:07.689451 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:07.689459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:07.689511 (+ 52us) server_negotiation.cc:407] Connection header received
0927 11:12:07.689595 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:07.689600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:07.689642 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:07.689740 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:07.690576 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:07.691947 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:07.692994 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:07.693354 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:07.693461 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:07.695599 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:07.695603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:07.695603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:07.695614 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:07.697241 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:07.697638 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:07.697641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:07.697641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:07.697730 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:07.697974 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:07.697975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:07.697976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:07.698182 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:07.698354 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:07.698428 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 11:12:07.698516 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260927 11:12:07.699718 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:07.706622 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:07.700981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60725 (local address 127.17.95.254:38957)
0927 11:12:07.701017 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:07.701019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:07.701139 (+ 120us) server_negotiation.cc:407] Connection header received
0927 11:12:07.701199 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:07.701200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:07.701209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:07.701262 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:07.701731 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:07.702218 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:07.702889 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:07.703163 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:07.703232 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:07.704296 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:07.704297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:07.704297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:07.704303 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:07.705536 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:07.705870 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:07.705871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:07.705872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:07.705919 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:07.706149 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:07.706150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:07.706150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:07.706250 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:07.706353 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:07.706463 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 11:12:07.706520 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:12:07.708315 6900 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:07.708776 6900 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:07.722723 6900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a80f819dc44891919c6cdaf81dafa1: Generated new TSK 0
I20260927 11:12:07.722896 6900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:07.729658 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6821
W20260927 11:12:07.733891 6806 connection.cc:535] client connection to 127.17.95.254:38957 recv error: Network error: failed to read from TLS socket (remote: 127.17.95.254:38957): Connection reset by peer (error 104)
2026-09-27T11:12:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:07 dist-test-slave-6j0v krb5kdc[6920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:07 dist-test-slave-6j0v krb5kdc[6920](info): set up 2 sockets
Sep 27 11:12:07 dist-test-slave-6j0v krb5kdc[6920](info): commencing operation
krb5kdc: starting...
W20260927 11:12:08.731197 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:08 dist-test-slave-6j0v krb5kdc[6920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:08.843524 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35763
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:35763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:08.920811 6933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:08.921005 6933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:08.921027 6933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:08.922106 6933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:08.922145 6933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:08.922159 6933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:08.922171 6933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:08.922183 6933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:08.923650 6933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:35763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:08.923899 6933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:08.924115 6933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:08.928787 6941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:08.928901 6939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:08.928897 6938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:08.929167 6933 server_base.cc:1034] running on GCE node
I20260927 11:12:08.929411 6933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:08.929708 6933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:08.930876 6933 hybrid_clock.cc:648] HybridClock initialized: now 1790507528930875 us; error 35 us; skew 500 ppm
Sep 27 11:12:08 dist-test-slave-6j0v krb5kdc[6920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507528, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:08.938578 6933 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:08.939006 6933 webserver.cc:460] Webserver started at http://127.17.95.254:36117/ using document root <none> and password file <none>
I20260927 11:12:08.939262 6933 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:08.939342 6933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:08.939491 6933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:08.940529 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "3a9012b573e54e3b9ee23a9a292b5203"
format_stamp: "Formatted at 2026-09-27 11:12:08 on dist-test-slave-6j0v"
server_key: "2bfa6411910d2bb69aa97ada31360689"
server_key_iv: "1c790a25d50c173fd45df0f9ed5aa216"
server_key_version: "encryptionkey@0"
I20260927 11:12:08.940904 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "3a9012b573e54e3b9ee23a9a292b5203"
format_stamp: "Formatted at 2026-09-27 11:12:08 on dist-test-slave-6j0v"
server_key: "2bfa6411910d2bb69aa97ada31360689"
server_key_iv: "1c790a25d50c173fd45df0f9ed5aa216"
server_key_version: "encryptionkey@0"
I20260927 11:12:08.942212 6933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:12:08.942974 6948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:08.943192 6933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:12:08.943327 6933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "3a9012b573e54e3b9ee23a9a292b5203"
format_stamp: "Formatted at 2026-09-27 11:12:08 on dist-test-slave-6j0v"
server_key: "2bfa6411910d2bb69aa97ada31360689"
server_key_iv: "1c790a25d50c173fd45df0f9ed5aa216"
server_key_version: "encryptionkey@0"
I20260927 11:12:08.943415 6933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:08.962692 6933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:08.964516 6933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:08.964701 6933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:08.968701 6933 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:35763
I20260927 11:12:08.968750 7000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:35763 every 8 connection(s)
I20260927 11:12:08.969111 6933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:08.969940 7001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:08.972200 7001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203: Bootstrap starting.
I20260927 11:12:08.972718 7001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:08.972941 7001 log.cc:824] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:08.973551 7001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203: No bootstrap required, opened a new log
I20260927 11:12:08.975378 7001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9012b573e54e3b9ee23a9a292b5203" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35763 } }
I20260927 11:12:08.975508 7001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:08.975530 7001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9012b573e54e3b9ee23a9a292b5203, State: Initialized, Role: FOLLOWER
I20260927 11:12:08.975670 7001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9012b573e54e3b9ee23a9a292b5203" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35763 } }
I20260927 11:12:08.975744 7001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:08.975764 7001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:08.975792 7001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:08.976573 7001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9012b573e54e3b9ee23a9a292b5203" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35763 } }
I20260927 11:12:08.976660 7001 leader_election.cc:302] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a9012b573e54e3b9ee23a9a292b5203; no voters:
I20260927 11:12:08.976770 7001 leader_election.cc:288] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:08.976845 7004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:08.977025 7004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [term 1 LEADER]: Becoming Leader. State: Replica: 3a9012b573e54e3b9ee23a9a292b5203, State: Running, Role: LEADER
I20260927 11:12:08.977061 7001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:08.977111 7004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9012b573e54e3b9ee23a9a292b5203" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35763 } }
I20260927 11:12:08.977548 7006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a9012b573e54e3b9ee23a9a292b5203. Latest consensus state: current_term: 1 leader_uuid: "3a9012b573e54e3b9ee23a9a292b5203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9012b573e54e3b9ee23a9a292b5203" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35763 } } }
I20260927 11:12:08.977625 7006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:08.977671 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 6933
I20260927 11:12:08.977790 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:08.977980 17791 external_mini_cluster.cc:1428] Setting key 01d04e3bbb27019cb08350f01b1c2ca3
I20260927 11:12:08.978055 7012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:08.978252 7005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a9012b573e54e3b9ee23a9a292b5203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9012b573e54e3b9ee23a9a292b5203" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35763 } } }
I20260927 11:12:08.978588 7012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:08.978579 7005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:08.980041 7012 catalog_manager.cc:1347] Generated new cluster ID: c1bf660473be4c61b387d572f4436ad3
I20260927 11:12:08.980091 7012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:08.984740 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:08.978421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48611 (local address 127.17.95.254:35763)
0927 11:12:08.979249 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:08.979258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:08.979513 (+ 255us) server_negotiation.cc:407] Connection header received
0927 11:12:08.979659 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:08.979667 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:08.979726 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:08.979869 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:08.980067 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:08.981442 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:08.982562 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:08.983176 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:08.983269 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:08.983978 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:08.983982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:08.983982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:08.984170 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:08.984204 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:08.984217 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:08.984288 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:08.984395 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 11:12:08.984419 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":58,"threads_started":1}
I20260927 11:12:08.985616 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:08.990355 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:08.987129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54379 (local address 127.17.95.254:35763)
0927 11:12:08.987162 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:08.987164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:08.987173 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:12:08.987227 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:08.987228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:08.987236 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:08.987291 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:08.987421 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:08.988116 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:08.989126 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:08.989430 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:08.989487 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:08.990108 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:08.990110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:08.990110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:08.990114 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:08.990126 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:08.990130 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:08.990182 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:08.990269 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 11:12:08.990279 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:08.992103 7012 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:08.992769 7012 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:09.001691 7012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a9012b573e54e3b9ee23a9a292b5203: Generated new TSK 0
I20260927 11:12:09.001855 7012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:09.004765 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 6933
W20260927 11:12:09.009064 6918 connection.cc:535] client connection to 127.17.95.254:35763 recv error: Network error: failed to read from TLS socket (remote: 127.17.95.254:35763): Connection reset by peer (error 104)
2026-09-27T11:12:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1275 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:09 dist-test-slave-6j0v krb5kdc[7032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:09 dist-test-slave-6j0v krb5kdc[7032](info): set up 2 sockets
Sep 27 11:12:09 dist-test-slave-6j0v krb5kdc[7032](info): commencing operation
krb5kdc: starting...
W20260927 11:12:10.015182 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:10 dist-test-slave-6j0v krb5kdc[7032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:10.129841 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35607
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:35719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:35607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:10.210691 7045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:10.210915 7045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:10.210951 7045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:10.212147 7045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:10.212198 7045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:10.212213 7045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:10.212227 7045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:10.212241 7045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:10.213783 7045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:35719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:35607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:35607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:10.214043 7045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:10.214272 7045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:10.218905 7053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:10.218921 7051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:10.218995 7045 server_base.cc:1034] running on GCE node
W20260927 11:12:10.218909 7050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:10.219425 7045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:10.219689 7045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:10.220854 7045 hybrid_clock.cc:648] HybridClock initialized: now 1790507530220834 us; error 36 us; skew 500 ppm
Sep 27 11:12:10 dist-test-slave-6j0v krb5kdc[7032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507530, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:10.223278 7045 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:10.223701 7045 webserver.cc:460] Webserver started at http://127.17.95.254:37627/ using document root <none> and password file <none>
I20260927 11:12:10.223912 7045 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:10.223953 7045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:10.224083 7045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:10.225059 7045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "aa6136042bb34f618f73ccb43671c3cc"
format_stamp: "Formatted at 2026-09-27 11:12:10 on dist-test-slave-6j0v"
server_key: "0072aa9db14afbab9082c40596849b94"
server_key_iv: "f5ff90eacb7058c7453fe98eda15c046"
server_key_version: "encryptionkey@0"
I20260927 11:12:10.225392 7045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "aa6136042bb34f618f73ccb43671c3cc"
format_stamp: "Formatted at 2026-09-27 11:12:10 on dist-test-slave-6j0v"
server_key: "0072aa9db14afbab9082c40596849b94"
server_key_iv: "f5ff90eacb7058c7453fe98eda15c046"
server_key_version: "encryptionkey@0"
I20260927 11:12:10.226643 7045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:12:10.227417 7060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:10.227581 7045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:10.227680 7045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "aa6136042bb34f618f73ccb43671c3cc"
format_stamp: "Formatted at 2026-09-27 11:12:10 on dist-test-slave-6j0v"
server_key: "0072aa9db14afbab9082c40596849b94"
server_key_iv: "f5ff90eacb7058c7453fe98eda15c046"
server_key_version: "encryptionkey@0"
I20260927 11:12:10.227756 7045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:10.230852 7045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:10.232955 7045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:10.233104 7045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:10.237059 7045 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:35607
I20260927 11:12:10.237092 7112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:35607 every 8 connection(s)
I20260927 11:12:10.237427 7045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:10.238301 7113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:10.240474 7113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc: Bootstrap starting.
I20260927 11:12:10.240980 7113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:10.241235 7113 log.cc:824] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:10.241868 7113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc: No bootstrap required, opened a new log
I20260927 11:12:10.243484 7113 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6136042bb34f618f73ccb43671c3cc" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35607 } }
I20260927 11:12:10.243609 7113 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:10.243641 7113 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa6136042bb34f618f73ccb43671c3cc, State: Initialized, Role: FOLLOWER
I20260927 11:12:10.243739 7113 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6136042bb34f618f73ccb43671c3cc" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35607 } }
I20260927 11:12:10.243842 7113 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:10.243877 7113 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:10.243933 7113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:10.243950 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7045
I20260927 11:12:10.244069 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:10.244277 17791 external_mini_cluster.cc:1428] Setting key 2a5880b79b60d181baa8ee2fbcaeb1be
I20260927 11:12:10.245052 7113 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6136042bb34f618f73ccb43671c3cc" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35607 } }
I20260927 11:12:10.245153 7113 leader_election.cc:302] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa6136042bb34f618f73ccb43671c3cc; no voters:
I20260927 11:12:10.245299 7113 leader_election.cc:288] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:10.245352 7118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:10.245561 7118 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [term 1 LEADER]: Becoming Leader. State: Replica: aa6136042bb34f618f73ccb43671c3cc, State: Running, Role: LEADER
I20260927 11:12:10.245595 7113 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:10.245723 7118 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6136042bb34f618f73ccb43671c3cc" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35607 } }
I20260927 11:12:10.246580 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:10.244793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37393 (local address 127.17.95.254:35607)
0927 11:12:10.245002 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:10.245011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:10.245044 (+ 33us) server_negotiation.cc:407] Connection header received
0927 11:12:10.245164 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:10.245172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:10.245202 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:10.245351 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:10.246057 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:10.246060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:10.246060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:10.246065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:10.246107 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:10.246120 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:10.246161 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:10.246337 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 11:12:10.246382 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260927 11:12:10.246899 7120 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa6136042bb34f618f73ccb43671c3cc. Latest consensus state: current_term: 1 leader_uuid: "aa6136042bb34f618f73ccb43671c3cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6136042bb34f618f73ccb43671c3cc" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35607 } } }
I20260927 11:12:10.247013 7120 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:10.247001 7119 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa6136042bb34f618f73ccb43671c3cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6136042bb34f618f73ccb43671c3cc" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 35607 } } }
I20260927 11:12:10.247519 7119 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:10.247720 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260927 11:12:10.247727 7133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:10.247769 7133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:10.247831 7134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:10.247968 7134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:10.249583 7134 catalog_manager.cc:1347] Generated new cluster ID: fbcd1c997df2491a861bddf3568f9930
I20260927 11:12:10.249646 7134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:10.250348 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:10.249276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50715 (local address 127.17.95.254:35607)
0927 11:12:10.249306 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:10.249308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:10.249448 (+ 140us) server_negotiation.cc:407] Connection header received
0927 11:12:10.249493 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:10.249493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:10.249499 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:10.249558 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:10.250094 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:10.250095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:10.250095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:10.250099 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:10.250121 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:10.250126 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:10.250183 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:10.250279 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 11:12:10.250292 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:12:10.262372 7134 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:10.262519 7134 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:10.264572 7134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa6136042bb34f618f73ccb43671c3cc: Generated new TSK 0
I20260927 11:12:10.264756 7134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:10.269140 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7045
2026-09-27T11:12:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:10 dist-test-slave-6j0v krb5kdc[7144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:10 dist-test-slave-6j0v krb5kdc[7144](info): set up 2 sockets
Sep 27 11:12:10 dist-test-slave-6j0v krb5kdc[7144](info): commencing operation
krb5kdc: starting...
W20260927 11:12:11.271243 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:11 dist-test-slave-6j0v krb5kdc[7144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:11.388175 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:37629
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:37629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:11.468835 7157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:11.469030 7157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:11.469054 7157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:11.470124 7157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:11.470167 7157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:11.470182 7157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:11.470196 7157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:11.470250 7157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:11.471643 7157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:37629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:37629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:11.471882 7157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:11.472084 7157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:11.476949 7162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:11.476999 7165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:11.476974 7163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:11.477094 7157 server_base.cc:1034] running on GCE node
I20260927 11:12:11.477459 7157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:11.477711 7157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:11.478858 7157 hybrid_clock.cc:648] HybridClock initialized: now 1790507531478847 us; error 27 us; skew 500 ppm
Sep 27 11:12:11 dist-test-slave-6j0v krb5kdc[7144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507531, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:11.481489 7157 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:11.481945 7157 webserver.cc:460] Webserver started at http://127.17.95.254:36209/ using document root <none> and password file <none>
I20260927 11:12:11.482156 7157 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:11.482240 7157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:11.482390 7157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:11.483479 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "6af43b710cc34ad8949ab254a657f330"
format_stamp: "Formatted at 2026-09-27 11:12:11 on dist-test-slave-6j0v"
server_key: "ddf0f496ebd84c1c9ff8eb3a2660948d"
server_key_iv: "03790061719aba7fa5005eb52a05d641"
server_key_version: "encryptionkey@0"
I20260927 11:12:11.483887 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "6af43b710cc34ad8949ab254a657f330"
format_stamp: "Formatted at 2026-09-27 11:12:11 on dist-test-slave-6j0v"
server_key: "ddf0f496ebd84c1c9ff8eb3a2660948d"
server_key_iv: "03790061719aba7fa5005eb52a05d641"
server_key_version: "encryptionkey@0"
I20260927 11:12:11.485162 7157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:12:11.485978 7172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:11.486163 7157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:12:11.486277 7157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "6af43b710cc34ad8949ab254a657f330"
format_stamp: "Formatted at 2026-09-27 11:12:11 on dist-test-slave-6j0v"
server_key: "ddf0f496ebd84c1c9ff8eb3a2660948d"
server_key_iv: "03790061719aba7fa5005eb52a05d641"
server_key_version: "encryptionkey@0"
I20260927 11:12:11.486354 7157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:11.497968 7157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:11.499912 7157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:11.500062 7157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:11.503949 7157 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:37629
I20260927 11:12:11.503980 7224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:37629 every 8 connection(s)
I20260927 11:12:11.504315 7157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:11.505098 7225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:11.507243 7225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330: Bootstrap starting.
I20260927 11:12:11.507772 7225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:11.507992 7225 log.cc:824] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:11.508630 7225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330: No bootstrap required, opened a new log
I20260927 11:12:11.510115 7225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } }
I20260927 11:12:11.510245 7225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:11.510267 7225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af43b710cc34ad8949ab254a657f330, State: Initialized, Role: FOLLOWER
I20260927 11:12:11.510414 7225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } }
I20260927 11:12:11.510660 7228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 11:12:11.510747 7228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } }
I20260927 11:12:11.510471 7225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:11.510897 7225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:11.510885 7228 leader_election.cc:302] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af43b710cc34ad8949ab254a657f330; no voters:
I20260927 11:12:11.510947 7225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:11.511059 7228 leader_election.cc:288] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260927 11:12:11.511503 7225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } }
I20260927 11:12:11.511588 7225 leader_election.cc:302] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af43b710cc34ad8949ab254a657f330; no voters:
I20260927 11:12:11.511627 7229 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260927 11:12:11.511662 7225 leader_election.cc:288] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:11.511843 7229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } } }
I20260927 11:12:11.511922 7229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:11.511970 7225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:11.511996 7228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:11.512044 7228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [term 1 LEADER]: Becoming Leader. State: Replica: 6af43b710cc34ad8949ab254a657f330, State: Running, Role: LEADER
I20260927 11:12:11.512136 7228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } }
I20260927 11:12:11.512470 7229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af43b710cc34ad8949ab254a657f330. Latest consensus state: current_term: 1 leader_uuid: "6af43b710cc34ad8949ab254a657f330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af43b710cc34ad8949ab254a657f330" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 37629 } } }
I20260927 11:12:11.512557 7229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:11.512770 7236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:11.512904 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7157
I20260927 11:12:11.513031 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:11.513202 17791 external_mini_cluster.cc:1428] Setting key f7dadebcc1f26636b5d2c1100c4abea7
I20260927 11:12:11.513819 7236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:11.515457 7236 catalog_manager.cc:1347] Generated new cluster ID: 4c92b6f0b4d84b6a8910f9e10a72de6b
I20260927 11:12:11.515511 7236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 11:12:11 dist-test-slave-6j0v krb5kdc[7144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:11.523828 7236 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:11.524501 7236 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:11.526320 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:11.513807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42327 (local address 127.17.95.254:37629)
0927 11:12:11.514549 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:11.514555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:11.514578 (+ 23us) server_negotiation.cc:407] Connection header received
0927 11:12:11.514656 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:11.514661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:11.514707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:11.514846 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:11.515754 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:11.518244 (+ 2490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:11.519241 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:11.519865 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:11.519946 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:11.522717 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:11.522721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:11.522721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:11.522732 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:11.524722 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:11.525142 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:11.525144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:11.525145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:11.525249 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:11.525451 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:11.525452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:11.525452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:11.525718 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:11.525833 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:11.525971 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 11:12:11.526100 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":73,"threads_started":1}
I20260927 11:12:11.527304 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:11.531970 7236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6af43b710cc34ad8949ab254a657f330: Generated new TSK 0
I20260927 11:12:11.532137 7236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:11.536191 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:11.529740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46227 (local address 127.17.95.254:37629)
0927 11:12:11.529775 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:11.529777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:11.529785 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:12:11.529802 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:11.529803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:11.529813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:11.529869 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:11.530497 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:11.531286 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:11.532318 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:11.532513 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:11.532564 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:11.533864 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:11.533865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:11.533865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:11.533870 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:11.535117 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:11.535503 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:11.535504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:11.535504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:11.535576 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:11.535777 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:11.535778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:11.535778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:11.535864 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:11.535938 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:11.536065 (+ 127us) server_negotiation.cc:299] Negotiation successful
0927 11:12:11.536124 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:12:11.538776 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7157
2026-09-27T11:12:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:11 dist-test-slave-6j0v krb5kdc[7256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:11 dist-test-slave-6j0v krb5kdc[7256](info): set up 2 sockets
Sep 27 11:12:11 dist-test-slave-6j0v krb5kdc[7256](info): commencing operation
krb5kdc: starting...
W20260927 11:12:12.551115 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:12 dist-test-slave-6j0v krb5kdc[7256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:12.657708 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:43469
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:12.733018 7269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:12.733233 7269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:12.733268 7269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:12.734299 7269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:12.734356 7269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:12.734375 7269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:12.734393 7269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:12.734418 7269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:12.735728 7269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:43469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:12.735960 7269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:12.736156 7269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:12.740628 7277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:12.740661 7275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:12.740737 7269 server_base.cc:1034] running on GCE node
W20260927 11:12:12.740681 7274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:12.740986 7269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:12.741250 7269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:12.742406 7269 hybrid_clock.cc:648] HybridClock initialized: now 1790507532742400 us; error 39 us; skew 500 ppm
Sep 27 11:12:12 dist-test-slave-6j0v krb5kdc[7256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507532, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:12.744643 7269 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:12.745062 7269 webserver.cc:460] Webserver started at http://127.17.95.254:43813/ using document root <none> and password file <none>
I20260927 11:12:12.745270 7269 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:12.745332 7269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:12.745471 7269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:12.746392 7269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "eca9f71062dd4f59ae759fc766c8edcf"
format_stamp: "Formatted at 2026-09-27 11:12:12 on dist-test-slave-6j0v"
server_key: "c229fc2392170d1cf28b103903753cdd"
server_key_iv: "84c415ff5901ce06aec6a8eda2f7e69e"
server_key_version: "encryptionkey@0"
I20260927 11:12:12.746723 7269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "eca9f71062dd4f59ae759fc766c8edcf"
format_stamp: "Formatted at 2026-09-27 11:12:12 on dist-test-slave-6j0v"
server_key: "c229fc2392170d1cf28b103903753cdd"
server_key_iv: "84c415ff5901ce06aec6a8eda2f7e69e"
server_key_version: "encryptionkey@0"
I20260927 11:12:12.748068 7269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 11:12:12.748831 7284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:12.749007 7269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 11:12:12.749142 7269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "eca9f71062dd4f59ae759fc766c8edcf"
format_stamp: "Formatted at 2026-09-27 11:12:12 on dist-test-slave-6j0v"
server_key: "c229fc2392170d1cf28b103903753cdd"
server_key_iv: "84c415ff5901ce06aec6a8eda2f7e69e"
server_key_version: "encryptionkey@0"
I20260927 11:12:12.749212 7269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:12.758333 7269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:12.760059 7269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:12.760200 7269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:12.764109 7269 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:43469
I20260927 11:12:12.764148 7336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:43469 every 8 connection(s)
I20260927 11:12:12.764499 7269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:12.765318 7337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:12.767499 7337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf: Bootstrap starting.
I20260927 11:12:12.768054 7337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:12.768327 7337 log.cc:824] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:12.768983 7337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf: No bootstrap required, opened a new log
I20260927 11:12:12.770525 7337 raft_consensus.cc:348] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca9f71062dd4f59ae759fc766c8edcf" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:12:12.770676 7337 raft_consensus.cc:374] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:12.770699 7337 raft_consensus.cc:729] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca9f71062dd4f59ae759fc766c8edcf, State: Initialized, Role: FOLLOWER
I20260927 11:12:12.770870 7337 consensus_queue.cc:260] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca9f71062dd4f59ae759fc766c8edcf" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:12:12.770931 7337 raft_consensus.cc:388] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:12.770977 7337 raft_consensus.cc:482] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:12.771023 7337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:12.771543 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7269
I20260927 11:12:12.771633 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:12.771816 17791 external_mini_cluster.cc:1428] Setting key e803d609b83d2736d8a13a13295f16f7
I20260927 11:12:12.771994 7337 raft_consensus.cc:504] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca9f71062dd4f59ae759fc766c8edcf" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:12:12.772107 7337 leader_election.cc:302] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eca9f71062dd4f59ae759fc766c8edcf; no voters:
I20260927 11:12:12.772279 7337 leader_election.cc:288] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:12.772356 7340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:12.772531 7337 sys_catalog.cc:564] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:12.772535 7340 raft_consensus.cc:686] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [term 1 LEADER]: Becoming Leader. State: Replica: eca9f71062dd4f59ae759fc766c8edcf, State: Running, Role: LEADER
I20260927 11:12:12.772698 7340 consensus_queue.cc:237] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca9f71062dd4f59ae759fc766c8edcf" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } }
I20260927 11:12:12.773046 7343 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eca9f71062dd4f59ae759fc766c8edcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca9f71062dd4f59ae759fc766c8edcf" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } } }
I20260927 11:12:12.773137 7343 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:12.773046 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader eca9f71062dd4f59ae759fc766c8edcf. Latest consensus state: current_term: 1 leader_uuid: "eca9f71062dd4f59ae759fc766c8edcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca9f71062dd4f59ae759fc766c8edcf" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 43469 } } }
I20260927 11:12:12.773190 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:12.773833 7351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:12.774453 7351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:12.775976 7351 catalog_manager.cc:1347] Generated new cluster ID: 888cbbc7efbc4576b0e6bb8a36cad1ad
I20260927 11:12:12.776026 7351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:12.778306 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:12.772272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39593 (local address 127.17.95.254:43469)
0927 11:12:12.773285 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:12.773294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:12.773323 (+ 29us) server_negotiation.cc:407] Connection header received
0927 11:12:12.773429 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:12.773435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:12.773490 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:12.773652 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:12.773897 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:12.775186 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:12.776200 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:12.776795 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:12.776908 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:12.777608 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:12.777612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:12.777612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:12.777816 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:12.777856 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:12.777873 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:12.777945 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:12.778011 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 11:12:12.778075 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":86,"threads_started":1}
I20260927 11:12:12.779203 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:12.782835 7351 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:12.783442 7351 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:12.784570 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:12.780724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38967 (local address 127.17.95.254:43469)
0927 11:12:12.780761 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:12.780763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:12.780823 (+ 60us) server_negotiation.cc:407] Connection header received
0927 11:12:12.780881 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:12.780881 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:12.780891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:12.780955 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:12.781076 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:12.782206 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:12.783277 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:12.783586 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:12.783662 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:12.784282 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:12.784283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:12.784283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:12.784313 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:12.784327 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:12.784332 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:12.784419 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:12.784467 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 11:12:12.784502 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:12:12.788710 7351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eca9f71062dd4f59ae759fc766c8edcf: Generated new TSK 0
I20260927 11:12:12.788864 7351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:12.792284 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7269
2026-09-27T11:12:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1252 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:12 dist-test-slave-6j0v krb5kdc[7368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:12 dist-test-slave-6j0v krb5kdc[7368](info): set up 2 sockets
Sep 27 11:12:12 dist-test-slave-6j0v krb5kdc[7368](info): commencing operation
krb5kdc: starting...
W20260927 11:12:13.795241 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:13 dist-test-slave-6j0v krb5kdc[7368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:13.910290 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36689
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:36689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:13.989519 7381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:13.989725 7381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:13.989748 7381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:13.990833 7381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:13.990866 7381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:13.990880 7381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:13.990892 7381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:13.990904 7381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:13.992343 7381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:36689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:36689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:13.992527 7381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:13.992687 7381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:13.997321 7387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:13.997421 7389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:13.997313 7386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:13.997583 7381 server_base.cc:1034] running on GCE node
I20260927 11:12:13.997859 7381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:13.998176 7381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:13.999336 7381 hybrid_clock.cc:648] HybridClock initialized: now 1790507533999323 us; error 45 us; skew 500 ppm
Sep 27 11:12:14 dist-test-slave-6j0v krb5kdc[7368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507534, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:14.007764 7381 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:14.008225 7381 webserver.cc:460] Webserver started at http://127.17.95.254:44515/ using document root <none> and password file <none>
I20260927 11:12:14.008404 7381 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:14.008446 7381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:14.008533 7381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:14.009518 7381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "48ae933f91e544ca9ee82d14e9d96ebb"
format_stamp: "Formatted at 2026-09-27 11:12:14 on dist-test-slave-6j0v"
server_key: "3a17808d9508746673b6902201a7cf4d"
server_key_iv: "c10dd9ebe6d2db7e3885e304a2d652ad"
server_key_version: "encryptionkey@0"
I20260927 11:12:14.009891 7381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "48ae933f91e544ca9ee82d14e9d96ebb"
format_stamp: "Formatted at 2026-09-27 11:12:14 on dist-test-slave-6j0v"
server_key: "3a17808d9508746673b6902201a7cf4d"
server_key_iv: "c10dd9ebe6d2db7e3885e304a2d652ad"
server_key_version: "encryptionkey@0"
I20260927 11:12:14.011211 7381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:12:14.011960 7396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:14.012132 7381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:12:14.012225 7381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "48ae933f91e544ca9ee82d14e9d96ebb"
format_stamp: "Formatted at 2026-09-27 11:12:14 on dist-test-slave-6j0v"
server_key: "3a17808d9508746673b6902201a7cf4d"
server_key_iv: "c10dd9ebe6d2db7e3885e304a2d652ad"
server_key_version: "encryptionkey@0"
I20260927 11:12:14.012276 7381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:14.015350 7381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:14.017180 7381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:14.017280 7381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:14.020865 7381 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:36689
I20260927 11:12:14.020910 7448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:36689 every 8 connection(s)
I20260927 11:12:14.021189 7381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:14.022116 7449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:14.024348 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7381
I20260927 11:12:14.024384 7449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb: Bootstrap starting.
I20260927 11:12:14.024447 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:14.024685 17791 external_mini_cluster.cc:1428] Setting key 103daaa7bf225e4c599cba082b8de567
I20260927 11:12:14.025050 7449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:14.025384 7449 log.cc:824] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:14.026005 7449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb: No bootstrap required, opened a new log
I20260927 11:12:14.027024 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:14.025183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33565 (local address 127.17.95.254:36689)
0927 11:12:14.025392 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:14.025401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:14.025468 (+ 67us) server_negotiation.cc:407] Connection header received
0927 11:12:14.025864 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:14.025871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:14.025895 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:14.026034 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:14.026632 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:14.026636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:14.026636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:14.026640 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:14.026686 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:14.026697 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:14.026780 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:14.026835 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 11:12:14.026874 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260927 11:12:14.028012 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:14.028501 7449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36689 } }
I20260927 11:12:14.028654 7449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 11:12:14.028698 7449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ae933f91e544ca9ee82d14e9d96ebb, State: Initialized, Role: FOLLOWER
I20260927 11:12:14.028808 7449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36689 } }
I20260927 11:12:14.028916 7449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 11:12:14.028949 7449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 11:12:14.029011 7449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:14.029682 7449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36689 } }
I20260927 11:12:14.029793 7449 leader_election.cc:302] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ae933f91e544ca9ee82d14e9d96ebb; no voters:
I20260927 11:12:14.029939 7449 leader_election.cc:288] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 11:12:14.029989 7459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:14.030121 7459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [term 1 LEADER]: Becoming Leader. State: Replica: 48ae933f91e544ca9ee82d14e9d96ebb, State: Running, Role: LEADER
I20260927 11:12:14.030167 7449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:14.030222 7459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36689 } }
I20260927 11:12:14.030541 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48ae933f91e544ca9ee82d14e9d96ebb. Latest consensus state: current_term: 1 leader_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36689 } } }
I20260927 11:12:14.030602 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:14.029415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50335 (local address 127.17.95.254:36689)
0927 11:12:14.029448 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:14.029449 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:14.029565 (+ 116us) server_negotiation.cc:407] Connection header received
0927 11:12:14.029633 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:14.029634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:14.029638 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:14.029686 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:14.030220 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:14.030221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:14.030222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 11:12:14.030225 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:14.030237 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 11:12:14.030241 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:14.030294 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:14.030483 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 11:12:14.030494 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:14.030623 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:14.030802 7460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ae933f91e544ca9ee82d14e9d96ebb" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 36689 } } }
I20260927 11:12:14.030896 7460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:14.031656 7466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:14.032087 7466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:14.033371 7466 catalog_manager.cc:1347] Generated new cluster ID: b08faa91b64344db9b8a576376f0e302
I20260927 11:12:14.033424 7466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:14.052907 7466 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:14.053079 7466 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:14.068740 7466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48ae933f91e544ca9ee82d14e9d96ebb: Generated new TSK 0
I20260927 11:12:14.068961 7466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:14.070796 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7381
2026-09-27T11:12:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1279 ms)
[----------] 18 tests from AuthTokenIssuingTest (22970 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:14 dist-test-slave-6j0v krb5kdc[7480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:14 dist-test-slave-6j0v krb5kdc[7480](info): set up 2 sockets
Sep 27 11:12:14 dist-test-slave-6j0v krb5kdc[7480](info): commencing operation
krb5kdc: starting...
W20260927 11:12:15.074815 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:15.188033 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40975
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40975,127.17.95.253:44145,127.17.95.252:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:15.264715 7493 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 11:12:15.264890 7493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:15.264905 7493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:15.264923 7493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:15.265975 7493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:15.266011 7493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:15.266023 7493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:15.266057 7493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:15.266086 7493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:15.267477 7493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40975,127.17.95.253:44145,127.17.95.252:42861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:15.267707 7493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:15.267915 7493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:15.272553 7499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:15.272547 7501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:15.272545 7498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:15.272897 7493 server_base.cc:1034] running on GCE node
I20260927 11:12:15.273156 7493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:15.273453 7493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:15.274626 7493 hybrid_clock.cc:648] HybridClock initialized: now 1790507535274599 us; error 36 us; skew 500 ppm
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:15.277398 7493 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:15.277860 7493 webserver.cc:460] Webserver started at http://127.17.95.254:38379/ using document root <none> and password file <none>
I20260927 11:12:15.278100 7493 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:15.278175 7493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:15.278312 7493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:15.279438 7493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "ac8093fb85e243d99da49d5534f3881b"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "8994a65b0f8daeb752b2528b5a5dd14c"
server_key_iv: "57269cb61f7bc887171a5879adb21aa8"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.279811 7493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "ac8093fb85e243d99da49d5534f3881b"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "8994a65b0f8daeb752b2528b5a5dd14c"
server_key_iv: "57269cb61f7bc887171a5879adb21aa8"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.281121 7493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:12:15.281889 7508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:15.282080 7493 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:12:15.282188 7493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "ac8093fb85e243d99da49d5534f3881b"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "8994a65b0f8daeb752b2528b5a5dd14c"
server_key_iv: "57269cb61f7bc887171a5879adb21aa8"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.282277 7493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:15.296967 7493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:15.298867 7493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:15.299037 7493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:15.302939 7493 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:40975
I20260927 11:12:15.302999 7560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:40975 every 8 connection(s)
I20260927 11:12:15.303387 7493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:15.304301 7561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:15.305713 7561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:15.312021 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7493
I20260927 11:12:15.312153 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:15.312330 17791 external_mini_cluster.cc:1428] Setting key a3be8c7125a7849d789878a17077fb66
I20260927 11:12:15.315016 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.306393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.17.95.254:40975)
0927 11:12:15.306539 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.306547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.306687 (+ 140us) server_negotiation.cc:407] Connection header received
0927 11:12:15.307242 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.307244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.307263 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.307386 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.307911 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.308735 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.309565 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.309866 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.309981 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.311805 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.311811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.311811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.311817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.313386 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.313855 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.313857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.313857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.313946 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.314194 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.314195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.314195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.314395 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.314490 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.314702 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.314879 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260927 11:12:15.315028 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.306319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40975 (local address 127.0.0.1:59214)
0927 11:12:15.306531 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.306572 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.307053 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.307394 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.307396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.307646 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.307816 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.307820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.308849 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.308850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.309472 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.309474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.309959 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.309961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.310022 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.310416 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.310423 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.311681 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.313499 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.313505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.313507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.313741 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.314017 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.314018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.314018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.314113 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.314470 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.314475 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.314537 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.314637 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.314803 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260927 11:12:15.315834 7561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:15.316207 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.316059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47798)
0927 11:12:15.316121 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.316174 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 11:12:15.316327 7511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
I20260927 11:12:15.316710 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.316627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47804)
0927 11:12:15.316671 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.316694 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 11:12:15.316848 7561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:44145: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.95.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab.
Entry for principal kudu/127.17.95.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab.
I20260927 11:12:15.336156 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:44145
--webserver_interface=127.17.95.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab
--principal=kudu/127.17.95.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40975,127.17.95.253:44145,127.17.95.252:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:12:15.366061 7561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } attempt: 1
I20260927 11:12:15.366495 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.366375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47820)
0927 11:12:15.366437 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.366468 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 11:12:15.366822 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.366717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47834)
0927 11:12:15.366774 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.366797 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 11:12:15.366899 7561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:44145: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
W20260927 11:12:15.412070 7568 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 11:12:15.412281 7568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:15.412297 7568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:15.412313 7568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:15.413419 7568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:15.413455 7568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:15.413468 7568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:15.413480 7568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:15.413492 7568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:15.414809 7568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40975,127.17.95.253:44145,127.17.95.252:42861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab
--principal=kudu/127.17.95.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:44145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.95.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:15.415020 7568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:15.415236 7568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:15.419605 7573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:15.419611 7576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:15.419682 7574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:15.419852 7568 server_base.cc:1034] running on GCE node
I20260927 11:12:15.420094 7568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:15.420353 7568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:15.421487 7568 hybrid_clock.cc:648] HybridClock initialized: now 1790507535421465 us; error 37 us; skew 500 ppm
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:15.423959 7568 init.cc:377] Logged in from keytab as kudu/127.17.95.253@KRBTEST.COM (short username kudu)
I20260927 11:12:15.424321 7568 webserver.cc:460] Webserver started at http://127.17.95.253:42045/ using document root <none> and password file <none>
I20260927 11:12:15.424484 7568 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:15.424520 7568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:15.424602 7568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:15.425535 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data/instance:
uuid: "444faa7eac994bf89d13ee97760fefdf"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "c5fc7d16601e7ecdd68ba356a7fbc9e4"
server_key_iv: "574bf8a655e97aac44009918ffb48f5f"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.425848 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal/instance:
uuid: "444faa7eac994bf89d13ee97760fefdf"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "c5fc7d16601e7ecdd68ba356a7fbc9e4"
server_key_iv: "574bf8a655e97aac44009918ffb48f5f"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.427062 7568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 11:12:15.427836 7583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:15.428007 7568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:12:15.428103 7568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal
uuid: "444faa7eac994bf89d13ee97760fefdf"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "c5fc7d16601e7ecdd68ba356a7fbc9e4"
server_key_iv: "574bf8a655e97aac44009918ffb48f5f"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.428190 7568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:15.435036 7561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } attempt: 2
I20260927 11:12:15.435475 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.435353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47836)
0927 11:12:15.435412 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.435447 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 11:12:15.435890 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.435804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47852)
0927 11:12:15.435849 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.435873 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 11:12:15.436002 7561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:44145: Network error: Client connection negotiation failed: client connection to 127.17.95.253:44145: connect: Connection refused (error 111)
I20260927 11:12:15.464434 7568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:15.466323 7568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:15.466499 7568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:15.470644 7568 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.253:44145
I20260927 11:12:15.470703 7635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.253:44145 every 8 connection(s)
I20260927 11:12:15.471031 7568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
I20260927 11:12:15.472059 7636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:15.473485 7636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:15.480674 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7568
I20260927 11:12:15.480755 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-1/wal/instance
I20260927 11:12:15.480937 17791 external_mini_cluster.cc:1428] Setting key efd6573c4a3454e7fca1897c8dd1e3ce
I20260927 11:12:15.481563 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.474102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40975 (local address 127.0.0.1:59220)
0927 11:12:15.474260 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.474299 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.474711 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.474915 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.474919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.475169 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.475345 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.475349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.475953 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.475954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.476664 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.476666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.476989 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.476990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.477080 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.477541 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.477548 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.478924 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.480244 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.480250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.480251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.480449 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.480661 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.480662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.480662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.480800 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.481087 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.481092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.481214 (+ 122us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.481294 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.481418 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260927 11:12:15.481588 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.474105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.17.95.254:40975)
0927 11:12:15.474163 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.474165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.474381 (+ 216us) server_negotiation.cc:407] Connection header received
0927 11:12:15.474785 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.474786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.474796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.474858 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.475415 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.475880 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.476730 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.476916 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.476964 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.479017 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.479018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.479019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.479024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.480180 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.480532 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.480533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.480534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.480593 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.480857 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.480858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.480858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.481002 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.481090 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.481339 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.481447 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 11:12:15.482080 7636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.252@KRBTEST.COM; defaulting to no policy
I20260927 11:12:15.489807 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.482363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47866)
0927 11:12:15.482405 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.482415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.482461 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.482747 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.482748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.482842 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.482886 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.482887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.484118 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.484119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.484559 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.484560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.485266 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.485267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.485316 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.485668 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.485672 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.486655 (+ 983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.488670 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.488670 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.488671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.488817 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.489072 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.489073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.489073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.489137 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.489488 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.489489 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.489554 (+ 65us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.489620 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.489746 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 11:12:15.489977 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.482455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.17.95.253:44145)
0927 11:12:15.482614 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.482619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.482647 (+ 28us) server_negotiation.cc:407] Connection header received
0927 11:12:15.482666 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.482667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.482680 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.482743 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.482957 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.483990 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.484668 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.485196 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.485270 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.486729 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.486736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.486736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.486742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.488585 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.488892 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.488894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.488895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.488957 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.489205 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.489205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.489206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.489366 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.489466 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.489655 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.489793 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260927 11:12:15.490520 7636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:15.490883 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.490734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38536)
0927 11:12:15.490796 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.490836 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 11:12:15.491048 7587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
I20260927 11:12:15.491480 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.491377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38544)
0927 11:12:15.491431 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.491458 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.252@KRBTEST.COM" created.
W20260927 11:12:15.491601 7636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:42861: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab.
Entry for principal kudu/127.17.95.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab.
I20260927 11:12:15.503769 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:42861
--webserver_interface=127.17.95.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab
--principal=kudu/127.17.95.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40975,127.17.95.253:44145,127.17.95.252:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:12:15.527133 7561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } attempt: 3
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:12:15.534674 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.527602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.17.95.253:44145)
0927 11:12:15.527678 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.527680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.527693 (+ 13us) server_negotiation.cc:407] Connection header received
0927 11:12:15.527775 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.527776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.527791 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.527870 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.528227 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.528788 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.529577 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.529805 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.529892 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.532103 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.532104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.532104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.532111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.533654 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.534018 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.534019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.534019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.534066 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.534249 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.534249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.534249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.534343 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.534392 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.534533 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.534590 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 11:12:15.534714 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.527545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47870)
0927 11:12:15.527620 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.527649 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.527702 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.527877 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.527878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.528055 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.528135 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.528136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.528910 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.528912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.529457 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.529459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.529892 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.529893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.529976 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.530489 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.530500 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.531999 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.533739 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.533740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.533741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.533944 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.534126 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.534127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.534127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.534191 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.534400 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.534401 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.534446 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.534512 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.534598 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 11:12:15.535194 7561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:15.535511 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.535427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38550)
0927 11:12:15.535460 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.535489 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 11:12:15.535792 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.535736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38564)
0927 11:12:15.535757 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.535780 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260927 11:12:15.535887 7561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:42861: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
I20260927 11:12:15.540798 7636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } attempt: 1
I20260927 11:12:15.541110 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.541007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38578)
0927 11:12:15.541057 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.541083 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 11:12:15.541406 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.541318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38592)
0927 11:12:15.541360 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.541383 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260927 11:12:15.541500 7636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:42861: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
I20260927 11:12:15.567022 7561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } attempt: 1
I20260927 11:12:15.567545 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.567383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38604)
0927 11:12:15.567457 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.567515 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 11:12:15.567898 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.567796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38616)
0927 11:12:15.567842 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.567868 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 11:12:15.567991 7561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:42861: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
W20260927 11:12:15.582940 7643 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 11:12:15.583174 7643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:15.583210 7643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:15.583242 7643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:15.584342 7643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:15.584398 7643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:15.584421 7643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:15.584450 7643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:15.584475 7643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:15.585906 7643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40975,127.17.95.253:44145,127.17.95.252:42861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab
--principal=kudu/127.17.95.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:42861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.95.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:15.586179 7643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:15.586385 7643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:15.590986 7649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:15.591085 7643 server_base.cc:1034] running on GCE node
W20260927 11:12:15.590981 7651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:15.591012 7648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:15.591475 7643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:15.591744 7643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:15.592916 7643 hybrid_clock.cc:648] HybridClock initialized: now 1790507535592902 us; error 23 us; skew 500 ppm
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:15.595362 7643 init.cc:377] Logged in from keytab as kudu/127.17.95.252@KRBTEST.COM (short username kudu)
I20260927 11:12:15.595754 7643 webserver.cc:460] Webserver started at http://127.17.95.252:38351/ using document root <none> and password file <none>
I20260927 11:12:15.595975 7643 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:15.596037 7643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:15.596194 7643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:15.597249 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data/instance:
uuid: "7ffcb995e4e547ada06360c0e057a43c"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "edaacc4543a1f2d1e99489fa8503f920"
server_key_iv: "aad97e2945e802d7bbec902c9ce569fe"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.597627 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal/instance:
uuid: "7ffcb995e4e547ada06360c0e057a43c"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "edaacc4543a1f2d1e99489fa8503f920"
server_key_iv: "aad97e2945e802d7bbec902c9ce569fe"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.598965 7643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:12:15.599766 7658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:15.599938 7643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 11:12:15.600039 7643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal
uuid: "7ffcb995e4e547ada06360c0e057a43c"
format_stamp: "Formatted at 2026-09-27 11:12:15 on dist-test-slave-6j0v"
server_key: "edaacc4543a1f2d1e99489fa8503f920"
server_key_iv: "aad97e2945e802d7bbec902c9ce569fe"
server_key_version: "encryptionkey@0"
I20260927 11:12:15.600134 7643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:15.609644 7636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } attempt: 2
I20260927 11:12:15.610090 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.609946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38624)
0927 11:12:15.610008 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.610051 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 11:12:15.610410 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.610330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38638)
0927 11:12:15.610357 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.610390 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260927 11:12:15.610502 7636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:42861: Network error: Client connection negotiation failed: client connection to 127.17.95.252:42861: connect: Connection refused (error 111)
I20260927 11:12:15.615343 7643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:15.617631 7643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:15.617748 7643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:15.621793 7643 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.252:42861
I20260927 11:12:15.621821 7710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.252:42861 every 8 connection(s)
I20260927 11:12:15.622184 7643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
I20260927 11:12:15.623208 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:15.624636 7711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:15.627786 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7643
I20260927 11:12:15.627869 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790507277271108-17791-0/minicluster-data/master-2/wal/instance
I20260927 11:12:15.628054 17791 external_mini_cluster.cc:1428] Setting key c780e66f698bd8fbc3bea3d0af29d30a
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:12:15.634001 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.625286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.17.95.254:40975)
0927 11:12:15.625346 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.625348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.625546 (+ 198us) server_negotiation.cc:407] Connection header received
0927 11:12:15.626055 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.626056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.626067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.626110 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.626691 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.627211 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.628325 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.628581 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.628663 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.631060 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.631061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.631062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.631085 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.632619 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.633052 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.633053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.633054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.633125 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.633367 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.633368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.633368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.633519 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.633602 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.633885 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.633944 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 11:12:15.634236 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.625257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40975 (local address 127.0.0.1:59236)
0927 11:12:15.625442 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.625473 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.625921 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.626210 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.626213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.626423 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.626592 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.626595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.627295 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.627296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.628245 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.628248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.628751 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.628752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.628881 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.629362 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.629379 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.630949 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.632732 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.632739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.632740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.632955 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.633191 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.633192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.633192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.633298 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.633593 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.633600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.633734 (+ 134us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.633868 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.634011 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260927 11:12:15.634773 7711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:15.636684 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.628442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.17.95.252:42861)
0927 11:12:15.628629 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.628633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.628767 (+ 134us) server_negotiation.cc:407] Connection header received
0927 11:12:15.628797 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.628799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.628817 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.628876 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.629443 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.630264 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.631247 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.631630 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.631708 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.633779 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.633784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.633785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.633790 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.635652 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.635957 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.635959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.635960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.636013 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.636213 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.636217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.636217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.636329 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.636420 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.636549 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.636602 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:15.641479 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.635019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:44145 (local address 127.0.0.1:47876)
0927 11:12:15.635131 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.635140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.635178 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.635299 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.635299 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.635464 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.635491 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.635492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.636164 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.636166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.636915 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.636917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.637333 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.637334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.637410 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.637738 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.637742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.638938 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.640585 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.640586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.640586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.640745 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.640928 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.640929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.640929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.640978 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.641211 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.641212 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.641248 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.641319 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.641399 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260927 11:12:15.641492 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.635099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.17.95.253:44145)
0927 11:12:15.635159 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.635160 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.635171 (+ 11us) server_negotiation.cc:407] Connection header received
0927 11:12:15.635235 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.635236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.635244 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.635296 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.635558 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.636100 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.637012 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.637264 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.637332 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.639014 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.639015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.639016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.639023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.640499 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.640811 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.640812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.640812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.640861 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.641042 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.641043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.641043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.641143 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.641207 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.641353 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.641416 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 11:12:15.641889 7711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:15.643128 7561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } attempt: 2
I20260927 11:12:15.644816 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.637429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.17.95.254:40975)
0927 11:12:15.637499 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.637501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.637726 (+ 225us) server_negotiation.cc:407] Connection header received
0927 11:12:15.637744 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.637745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.637753 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.637795 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.638451 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.638931 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.639607 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.639803 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.639887 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.641947 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.641948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.641948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.641954 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.643579 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.643937 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.643938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.643938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.644006 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.644213 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.644214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.644214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.644325 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.644458 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.644639 (+ 181us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.644714 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:12:15.649283 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.642081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38660)
0927 11:12:15.642117 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.642150 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.642421 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.642536 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.642537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.642675 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.642705 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.642706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.643526 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.643527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.644234 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.644236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.644580 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.644581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.644642 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.645096 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.645101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.646328 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.647908 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.647908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.647909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.648076 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.648533 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.648533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.648534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.648634 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.648889 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.648890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.648937 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.649016 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.649111 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:12:15.649330 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.642172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.17.95.252:42861)
0927 11:12:15.642206 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.642208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.642424 (+ 216us) server_negotiation.cc:407] Connection header received
0927 11:12:15.642477 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.642477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.642486 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.642532 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.642776 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.643449 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.644301 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.644502 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.644580 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.646411 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.646412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.646412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.646418 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.647829 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.648397 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.648398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.648398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.648457 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.648699 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.648700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.648700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.648808 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.648893 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.649053 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.649172 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":19}
I20260927 11:12:15.651676 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: Bootstrap starting.
I20260927 11:12:15.651809 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.643369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38662)
0927 11:12:15.643409 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.643418 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.643449 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.643770 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.643771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.643896 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.643945 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.643946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.644793 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.644794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.645421 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.645424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.645884 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.645886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.645939 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.646768 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.646772 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.648045 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.650537 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.650538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.650539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.650771 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.651031 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.651031 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.651032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.651146 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.651456 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.651457 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.651501 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.651598 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.651694 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 11:12:15.652103 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.643404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.17.95.252:42861)
0927 11:12:15.643671 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.643673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.643677 (+ 4us) server_negotiation.cc:407] Connection header received
0927 11:12:15.643690 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.643691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.643697 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.643764 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.644054 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.644706 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.645508 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.645802 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.646064 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.648151 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.648152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.648153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.648159 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.650406 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.650843 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.650844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.650845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.650923 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.651231 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.651232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.651232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.651374 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.651463 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.651859 (+ 396us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.651944 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":62,"threads_started":1}
I20260927 11:12:15.652413 7711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:15.652712 7711 log.cc:824] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:15.653668 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.645576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.17.95.253:44145)
0927 11:12:15.645621 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.645623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.645631 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:12:15.645653 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.645654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.645663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.645723 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.646258 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.646659 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.647565 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.647864 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.647902 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.650735 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.650736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.650737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.650743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.652501 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.652887 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.652888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.652888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.652948 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.653195 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.653196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.653196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.653324 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.653475 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.653511 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.653576 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:12:15.654093 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: No bootstrap required, opened a new log
I20260927 11:12:15.654495 7561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b: Bootstrap starting.
I20260927 11:12:15.655211 7561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:15.655519 7561 log.cc:824] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:15.656354 7561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b: No bootstrap required, opened a new log
I20260927 11:12:15.656527 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:15.656510 7711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.656647 7711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ffcb995e4e547ada06360c0e057a43c, State: Initialized, Role: FOLLOWER
I20260927 11:12:15.656737 7711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.657011 7711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:15.657544 7535 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ac8093fb85e243d99da49d5534f3881b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59246
I20260927 11:12:15.658456 7719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.658558 7719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.658610 7561 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.658746 7561 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac8093fb85e243d99da49d5534f3881b, State: Initialized, Role: FOLLOWER
I20260927 11:12:15.658825 7561 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.659153 7561 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:15.659209 7535 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 11:12:15.659296 7535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 0 FOLLOWER]: Advancing to term 1
W20260927 11:12:15.659307 7730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:15.659356 7730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:15.659935 7535 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.660152 7535 leader_election.cc:288] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [CANDIDATE]: Term 1 election: Requested vote from peers 444faa7eac994bf89d13ee97760fefdf (127.17.95.253:44145), 7ffcb995e4e547ada06360c0e057a43c (127.17.95.252:42861)
I20260927 11:12:15.660576 7685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac8093fb85e243d99da49d5534f3881b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7ffcb995e4e547ada06360c0e057a43c"
I20260927 11:12:15.660714 7685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:15.659015 7643 master_runner.cc:186] Error getting master registration for 127.17.95.254:40975: OK, instance_id { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" instance_seqno: 1790507535301957 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 11:12:15.661455 7643 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 11:12:15.661576 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.661597 7685 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac8093fb85e243d99da49d5534f3881b in term 1.
I20260927 11:12:15.661648 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.661875 7512 leader_election.cc:302] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ffcb995e4e547ada06360c0e057a43c, ac8093fb85e243d99da49d5534f3881b; no voters:
I20260927 11:12:15.662032 7732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:15.662209 7732 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [term 1 LEADER]: Becoming Leader. State: Replica: ac8093fb85e243d99da49d5534f3881b, State: Running, Role: LEADER
I20260927 11:12:15.662298 7732 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.662680 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac8093fb85e243d99da49d5534f3881b. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.662773 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:15.662411 7493 master_runner.cc:186] Error getting master registration for 127.17.95.253:44145: OK, instance_id { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" instance_seqno: 1790507535469500 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 11:12:15.665002 7493 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 11:12:15.665551 7743 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:15.665604 7743 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:15.670285 7752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:15.670483 7752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:15.676621 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.661366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.17.95.253:44145)
0927 11:12:15.661401 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.661403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.665756 (+ 4353us) server_negotiation.cc:407] Connection header received
0927 11:12:15.665846 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.665847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.665857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.665915 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.666924 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.667583 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.668560 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.668820 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.668863 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.670470 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.670471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.670472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.670478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.672420 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.672839 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.672841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.672842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.672906 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.675307 (+ 2401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.675309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.675309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.675439 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.675530 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.676419 (+ 889us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.676497 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:15.679487 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.666004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.17.95.254:40975)
0927 11:12:15.666042 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.666043 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.669384 (+ 3341us) server_negotiation.cc:407] Connection header received
0927 11:12:15.669496 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.669497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.669507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.669540 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.671311 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.671989 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.673014 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.673284 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.673326 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.674792 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.674793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.674793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.674799 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.676818 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.678573 (+ 1755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.678574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.678574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.678645 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.678852 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.678853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.678853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.678968 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.679046 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.679306 (+ 260us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.679377 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:12:15.680109 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.661951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.17.95.252:42861)
0927 11:12:15.661993 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.661994 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.667005 (+ 5011us) server_negotiation.cc:407] Connection header received
0927 11:12:15.667153 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.667154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.667164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.667190 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.668336 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.669040 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.670515 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.670827 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.670867 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.676226 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.676227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.676228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.676234 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.678240 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.678830 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.678831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.678832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.678895 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.679185 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.679185 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.679186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.679310 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.679404 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.679924 (+ 520us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.680021 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 11:12:15.683883 7685 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [term 1 FOLLOWER]: Refusing update from remote peer ac8093fb85e243d99da49d5534f3881b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:12:15.684283 7749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:12:15.685817 7719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac8093fb85e243d99da49d5534f3881b. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.685915 7719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.687489 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.687572 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:15.687706 7755 mvcc.cc:205] Tried to move back new op lower bound from 7333918866159095808 to 7333918866073391104. Current Snapshot: MvccSnapshot[applied={T|T < 7333918866159095808}]
I20260927 11:12:15.687740 7719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.687832 7719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.688242 7757 mvcc.cc:205] Tried to move back new op lower bound from 7333918866159095808 to 7333918866073391104. Current Snapshot: MvccSnapshot[applied={T|T < 7333918866159095808}]
I20260927 11:12:15.691772 7752 catalog_manager.cc:1347] Generated new cluster ID: 292bc107bd77430c81e9532302748714
I20260927 11:12:15.691821 7752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:15.701613 7636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } attempt: 3
I20260927 11:12:15.705399 7752 catalog_manager.cc:1370] Generated new certificate authority record
Sep 27 11:12:15 dist-test-slave-6j0v krb5kdc[7480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507535, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:12:15.707132 7752 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:15.709128 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.701844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:42861 (local address 127.0.0.1:38678)
0927 11:12:15.701885 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:15.701895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:15.701934 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:15.702066 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:15.702066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:15.702251 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:15.702293 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.702295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.703163 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.703165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.703879 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:15.703882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.704349 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.704350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:15.704415 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.704868 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:15.704873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:15.706299 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:15.708175 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.708177 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.708177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.708373 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.708551 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:15.708552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:15.708552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:15.708604 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:15.708865 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:15.708866 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:15.708904 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 11:12:15.708959 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 11:12:15.709040 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 11:12:15.709376 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:15.701881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.17.95.252:42861)
0927 11:12:15.701914 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.701915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.701922 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:12:15.701981 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.701982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.702003 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.702062 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.702349 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.703098 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.703949 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:15.704254 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:15.704325 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:15.706391 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:15.706392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:15.706392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:15.706399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:15.708085 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.708443 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.708444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.708444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.708491 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:15.708685 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:15.708685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:15.708686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:15.708793 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:15.708843 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:15.709235 (+ 392us) server_negotiation.cc:299] Negotiation successful
0927 11:12:15.709301 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:15.710813 7636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: Bootstrap starting.
I20260927 11:12:15.711529 7636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:15.711819 7636 log.cc:824] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:15.712550 7636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: No bootstrap required, opened a new log
I20260927 11:12:15.714457 7636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.714589 7636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444faa7eac994bf89d13ee97760fefdf, State: Initialized, Role: FOLLOWER
I20260927 11:12:15.714677 7636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } }
I20260927 11:12:15.714962 7636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:15.715102 7609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:15.715935 7609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [term 1 FOLLOWER]: Refusing update from remote peer ac8093fb85e243d99da49d5534f3881b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:12:15.716094 7610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac8093fb85e243d99da49d5534f3881b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "444faa7eac994bf89d13ee97760fefdf"
I20260927 11:12:15.716189 7732 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [LEADER]: Connected to new peer: Peer: permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:12:15.716796 7610 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac8093fb85e243d99da49d5534f3881b in term 1.
W20260927 11:12:15.717090 7589 tablet.cc:2369] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 11:12:15.719079 7762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.719244 7762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.719336 7762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac8093fb85e243d99da49d5534f3881b. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.719435 7762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.719496 7762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.719575 7762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:15.722515 7752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b: Generated new TSK 0
I20260927 11:12:15.722913 7752 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 11:12:15.723846 7773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:15.723948 7773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:15.724447 7775 mvcc.cc:205] Tried to move back new op lower bound from 7333918866238541824 to 7333918866073391104. Current Snapshot: MvccSnapshot[applied={T|T < 7333918866159095808}]
I20260927 11:12:15.725191 7756 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac8093fb85e243d99da49d5534f3881b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac8093fb85e243d99da49d5534f3881b" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40975 } } peers { permanent_uuid: "444faa7eac994bf89d13ee97760fefdf" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 44145 } } peers { permanent_uuid: "7ffcb995e4e547ada06360c0e057a43c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 42861 } } }
I20260927 11:12:15.725272 7756 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8093fb85e243d99da49d5534f3881b [sys.catalog]: This master's current role is: LEADER
W20260927 11:12:15.750891 7716 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.749212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.17.95.252:42861)
0927 11:12:15.749257 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.749259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.750198 (+ 939us) server_negotiation.cc:407] Connection header received
0927 11:12:15.750310 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.750311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.750323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.750353 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.750821 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38684: BlockingRecv error: recv got EOF from 127.0.0.1:38684 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 11:12:15.751842 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.749588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.17.95.253:44145)
0927 11:12:15.749624 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.749626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.751312 (+ 1686us) server_negotiation.cc:407] Connection header received
0927 11:12:15.751406 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.751407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.751417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.751446 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.751784 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47910: BlockingRecv error: recv got EOF from 127.0.0.1:47910 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 11:12:15.752785 7564 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.749919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.17.95.254:40975)
0927 11:12:15.749954 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.749956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.752177 (+ 2221us) server_negotiation.cc:407] Connection header received
0927 11:12:15.752346 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.752347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.752357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.752385 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.752727 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59268: BlockingRecv error: recv got EOF from 127.0.0.1:59268 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 11:12:15.755170 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.754472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.17.95.253:44145)
0927 11:12:15.754513 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.754514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.754648 (+ 134us) server_negotiation.cc:407] Connection header received
0927 11:12:15.754765 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.754766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.754781 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.754810 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.755132 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47924: BlockingRecv error: recv got EOF from 127.0.0.1:47924 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 11:12:15.758163 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.757365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.17.95.253:44145)
0927 11:12:15.757396 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.757398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.757726 (+ 328us) server_negotiation.cc:407] Connection header received
0927 11:12:15.757813 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.757814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.757823 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.757849 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.758125 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47938: BlockingRecv error: recv got EOF from 127.0.0.1:47938 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 11:12:15.761977 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.761246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.17.95.253:44145)
0927 11:12:15.761287 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.761288 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.761504 (+ 216us) server_negotiation.cc:407] Connection header received
0927 11:12:15.761589 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.761590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.761598 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.761623 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.761942 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47948: BlockingRecv error: recv got EOF from 127.0.0.1:47948 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 11:12:15.767941 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.767078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.17.95.253:44145)
0927 11:12:15.767159 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.767160 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.767412 (+ 252us) server_negotiation.cc:407] Connection header received
0927 11:12:15.767534 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.767535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.767547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.767574 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.767906 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47954: BlockingRecv error: recv got EOF from 127.0.0.1:47954 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260927 11:12:15.777805 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.777030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.17.95.253:44145)
0927 11:12:15.777064 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.777066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.777301 (+ 235us) server_negotiation.cc:407] Connection header received
0927 11:12:15.777393 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.777393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.777403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.777439 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.777764 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47956: BlockingRecv error: recv got EOF from 127.0.0.1:47956 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 11:12:15.796108 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.795223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.17.95.253:44145)
0927 11:12:15.795265 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.795267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.795541 (+ 274us) server_negotiation.cc:407] Connection header received
0927 11:12:15.795640 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.795641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.795653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.795695 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.796056 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47972: BlockingRecv error: recv got EOF from 127.0.0.1:47972 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 11:12:15.830333 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.829599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.17.95.253:44145)
0927 11:12:15.829645 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.829647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.829768 (+ 121us) server_negotiation.cc:407] Connection header received
0927 11:12:15.829853 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.829855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.829866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.829927 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.830246 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47984: BlockingRecv error: recv got EOF from 127.0.0.1:47984 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 11:12:15.896716 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:15.895976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47992 (local address 127.17.95.253:44145)
0927 11:12:15.896048 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:15.896051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:15.896105 (+ 54us) server_negotiation.cc:407] Connection header received
0927 11:12:15.896172 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:15.896174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:15.896187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:15.896277 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:15.896604 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47992: BlockingRecv error: recv got EOF from 127.0.0.1:47992 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 11:12:16.026430 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:16.025744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.17.95.253:44145)
0927 11:12:16.025827 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.025829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.025877 (+ 48us) server_negotiation.cc:407] Connection header received
0927 11:12:16.025941 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.025942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.025952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.025999 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:16.026344 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47996: BlockingRecv error: recv got EOF from 127.0.0.1:47996 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260927 11:12:16.284288 7640 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:16.283689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.17.95.253:44145)
0927 11:12:16.283763 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.283765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.283804 (+ 39us) server_negotiation.cc:407] Connection header received
0927 11:12:16.283828 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.283829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.283839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.283882 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:16.284231 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48008: BlockingRecv error: recv got EOF from 127.0.0.1:48008 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 11:12:16.659803 7730 catalog_manager.cc:1259] Loaded cluster ID: 292bc107bd77430c81e9532302748714
I20260927 11:12:16.659911 7730 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: loading cluster ID for follower catalog manager: success
I20260927 11:12:16.660867 7730 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: acquiring CA information for follower catalog manager: success
I20260927 11:12:16.661195 7730 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ffcb995e4e547ada06360c0e057a43c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:12:16.724444 7773 catalog_manager.cc:1259] Loaded cluster ID: 292bc107bd77430c81e9532302748714
I20260927 11:12:16.724529 7773 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: loading cluster ID for follower catalog manager: success
I20260927 11:12:16.725294 7773 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: acquiring CA information for follower catalog manager: success
I20260927 11:12:16.725544 7773 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 444faa7eac994bf89d13ee97760fefdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:12:16.800130 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:16.797482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48020 (local address 127.17.95.253:44145)
0927 11:12:16.797599 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.797601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.797679 (+ 78us) server_negotiation.cc:407] Connection header received
0927 11:12:16.797709 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.797710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.797762 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.797835 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:16.797977 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.798470 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.799309 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.799525 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.799585 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:16.799764 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:16.799874 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:16.799945 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:16.799982 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 11:12:16.799998 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260927 11:12:16.804430 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:16.802023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.17.95.252:42861)
0927 11:12:16.802124 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.802126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.802156 (+ 30us) server_negotiation.cc:407] Connection header received
0927 11:12:16.802196 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.802196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.802244 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.802295 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:16.802414 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.802866 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.803821 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.804006 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.804058 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:16.804182 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:16.804273 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:16.804329 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:16.804359 (+ 30us) server_negotiation.cc:299] Negotiation successful
0927 11:12:16.804374 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":62,"threads_started":1}
I20260927 11:12:16.808784 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:16.805933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.17.95.252:42861)
0927 11:12:16.805976 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.805977 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.806356 (+ 379us) server_negotiation.cc:407] Connection header received
0927 11:12:16.806451 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.806452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.806463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.806511 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:16.806636 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.807276 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.808056 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.808349 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.808408 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:16.808529 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:16.808558 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:16.808606 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:16.808691 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 11:12:16.808721 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:12:16.808954 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:16.805869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.17.95.253:44145)
0927 11:12:16.806011 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.806013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.806134 (+ 121us) server_negotiation.cc:407] Connection header received
0927 11:12:16.806169 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.806170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.806178 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.806235 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:16.806341 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.807062 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.808179 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.808463 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.808524 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:16.808672 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:16.808705 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:16.808745 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:16.808856 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 11:12:16.808882 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260927 11:12:16.809751 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:16.805919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.17.95.254:40975)
0927 11:12:16.806056 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:16.806058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:16.806066 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:12:16.806107 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:16.806108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:16.806169 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:16.806240 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:16.806407 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.806981 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.808563 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:16.808881 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:16.808947 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:16.809411 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:16.809515 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:16.809587 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:16.809648 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 11:12:16.809671 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":50,"threads_started":1}
I20260927 11:12:16.811395 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7493
I20260927 11:12:16.816566 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7568
I20260927 11:12:16.821807 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7643
2026-09-27T11:12:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2751 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 11:12:16 dist-test-slave-6j0v krb5kdc[7863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 11:12:16 dist-test-slave-6j0v krb5kdc[7863](info): set up 2 sockets
Sep 27 11:12:16 dist-test-slave-6j0v krb5kdc[7863](info): commencing operation
krb5kdc: starting...
W20260927 11:12:17.835399 17791 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/test-user.keytab.
Sep 27 11:12:17 dist-test-slave-6j0v krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T11:12:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T11:12:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.95.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
Entry for principal kudu/127.17.95.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab.
I20260927 11:12:17.946631 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40307
--webserver_interface=127.17.95.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40307,127.17.95.253:33691,127.17.95.252:41155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 11:12:18.024796 7876 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 11:12:18.025000 7876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:18.025018 7876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:18.025039 7876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:18.026103 7876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:18.026136 7876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:18.026150 7876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:18.026162 7876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:18.026176 7876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:18.027607 7876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40307,127.17.95.253:33691,127.17.95.252:41155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.254.keytab
--principal=kudu/127.17.95.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.254:40307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.95.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:18.027797 7876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:18.027964 7876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:18.032516 7881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:18.032505 7882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:18.032608 7884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:18.032987 7876 server_base.cc:1034] running on GCE node
I20260927 11:12:18.033296 7876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:18.033584 7876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:18.034741 7876 hybrid_clock.cc:648] HybridClock initialized: now 1790507538034726 us; error 24 us; skew 500 ppm
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:18.037420 7876 init.cc:377] Logged in from keytab as kudu/127.17.95.254@KRBTEST.COM (short username kudu)
I20260927 11:12:18.037856 7876 webserver.cc:460] Webserver started at http://127.17.95.254:44301/ using document root <none> and password file <none>
I20260927 11:12:18.038090 7876 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:18.038161 7876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:18.038327 7876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:18.039517 7876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data/instance:
uuid: "ad54ed5e3472409fb2ab9f99aa690c27"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "28b3c4b974012d196a517c24c7f07582"
server_key_iv: "a0a5c2413d174b6f0080e23d909514bc"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.039897 7876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal/instance:
uuid: "ad54ed5e3472409fb2ab9f99aa690c27"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "28b3c4b974012d196a517c24c7f07582"
server_key_iv: "a0a5c2413d174b6f0080e23d909514bc"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.041401 7876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 11:12:18.042192 7891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:18.042387 7876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 11:12:18.042498 7876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal
uuid: "ad54ed5e3472409fb2ab9f99aa690c27"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "28b3c4b974012d196a517c24c7f07582"
server_key_iv: "a0a5c2413d174b6f0080e23d909514bc"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.042577 7876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:18.054210 7876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:18.056707 7876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:18.056879 7876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:18.061167 7876 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.254:40307
I20260927 11:12:18.061199 7943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.254:40307 every 8 connection(s)
I20260927 11:12:18.061558 7876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/data/info.pb
I20260927 11:12:18.062472 7944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:18.064034 7944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:18.070884 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7876
I20260927 11:12:18.070982 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-0/wal/instance
I20260927 11:12:18.071219 17791 external_mini_cluster.cc:1428] Setting key 0299ee935e2b0733407b560eedda5fa8
I20260927 11:12:18.073393 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.064666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40307 (local address 127.0.0.1:35546)
0927 11:12:18.064856 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.064897 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.065377 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.065624 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.065626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.065871 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.066027 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.066031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.066968 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.066969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.067715 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.067717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.068167 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.068168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.068236 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.068751 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.068759 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.070135 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.071857 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.071864 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.071866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.072157 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.072396 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.072397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.072397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.072513 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.072957 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.072962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.073020 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.073099 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.073234 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260927 11:12:18.073483 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.064735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.17.95.254:40307)
0927 11:12:18.064893 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.064900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.064974 (+ 74us) server_negotiation.cc:407] Connection header received
0927 11:12:18.065497 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.065499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.065514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.065593 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.066104 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.066886 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.067785 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.068074 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.068149 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.070210 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.070216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.070216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.070222 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.071777 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.072252 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.072254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.072255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.072329 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.072594 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.072595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.072595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.072880 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.072966 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.073163 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.073341 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260927 11:12:18.074319 7944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:18.074648 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.074514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34042)
0927 11:12:18.074564 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.074614 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 11:12:18.074792 7894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
I20260927 11:12:18.075237 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.075127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34052)
0927 11:12:18.075162 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.075203 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260927 11:12:18.075410 7944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:33691: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.95.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab.
Entry for principal kudu/127.17.95.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab.
I20260927 11:12:18.095996 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:33691
--webserver_interface=127.17.95.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab
--principal=kudu/127.17.95.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40307,127.17.95.253:33691,127.17.95.252:41155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:12:18.124646 7944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } attempt: 1
I20260927 11:12:18.125118 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.124979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34056)
0927 11:12:18.125037 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.125079 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 11:12:18.125489 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.125391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34060)
0927 11:12:18.125443 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.125469 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 11:12:18.125602 7944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:33691: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
W20260927 11:12:18.175135 7951 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 11:12:18.175379 7951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:18.175407 7951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:18.175457 7951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:18.176513 7951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:18.176576 7951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:18.176612 7951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:18.176636 7951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:18.176682 7951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:18.178083 7951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40307,127.17.95.253:33691,127.17.95.252:41155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.253.keytab
--principal=kudu/127.17.95.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.253:33691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.95.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:18.178333 7951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:18.178562 7951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 11:12:18.183211 7951 server_base.cc:1034] running on GCE node
W20260927 11:12:18.183167 7959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:18.183169 7956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:18.183173 7957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:18.183530 7951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:18.183782 7951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:18.184934 7951 hybrid_clock.cc:648] HybridClock initialized: now 1790507538184932 us; error 36 us; skew 500 ppm
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:18.187320 7951 init.cc:377] Logged in from keytab as kudu/127.17.95.253@KRBTEST.COM (short username kudu)
I20260927 11:12:18.187810 7951 webserver.cc:460] Webserver started at http://127.17.95.253:38129/ using document root <none> and password file <none>
I20260927 11:12:18.188045 7951 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:18.188108 7951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:18.188237 7951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:18.189226 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data/instance:
uuid: "efb020b941a642be8f57b9b4a63e9741"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "81d7004b2ad2dc9c7ed57df313019542"
server_key_iv: "be00798a82d14e09f24e66d07f0526b5"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.189558 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal/instance:
uuid: "efb020b941a642be8f57b9b4a63e9741"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "81d7004b2ad2dc9c7ed57df313019542"
server_key_iv: "be00798a82d14e09f24e66d07f0526b5"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.190832 7951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 11:12:18.191529 7966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:18.191687 7951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:18.191776 7951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal
uuid: "efb020b941a642be8f57b9b4a63e9741"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "81d7004b2ad2dc9c7ed57df313019542"
server_key_iv: "be00798a82d14e09f24e66d07f0526b5"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.191854 7951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:18.193755 7944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } attempt: 2
I20260927 11:12:18.194119 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.193974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34066)
0927 11:12:18.194051 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.194087 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 11:12:18.194463 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.194362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34068)
0927 11:12:18.194410 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.194438 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 11:12:18.194561 7944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.253:33691: Network error: Client connection negotiation failed: client connection to 127.17.95.253:33691: connect: Connection refused (error 111)
I20260927 11:12:18.221308 7951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:18.222970 7951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:18.223138 7951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:18.227317 7951 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.253:33691
I20260927 11:12:18.227591 8018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.253:33691 every 8 connection(s)
I20260927 11:12:18.227774 7951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/data/info.pb
I20260927 11:12:18.228965 8019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:18.230342 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 7951
I20260927 11:12:18.230448 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-1/wal/instance
I20260927 11:12:18.230625 17791 external_mini_cluster.cc:1428] Setting key abfd2a6100f8f6b654ff57d9392bbf68
I20260927 11:12:18.230544 8019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.95.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.95.252@KRBTEST.COM" created.
I20260927 11:12:18.240790 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.231296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.17.95.254:40307)
0927 11:12:18.231380 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.231382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.231612 (+ 230us) server_negotiation.cc:407] Connection header received
0927 11:12:18.232289 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.232290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.232306 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.232379 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.232910 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.233443 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.234848 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.235134 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.235213 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.237598 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.237599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.237600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.237608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.239232 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.239674 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.239675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.239676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.239742 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.240021 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.240022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.240022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.240208 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.240296 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.240601 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.240694 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 11:12:18.240962 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.231263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40307 (local address 127.0.0.1:35554)
0927 11:12:18.231484 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.231527 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.232177 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.232427 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.232430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.232653 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.232830 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.232834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.233543 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.233545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.234751 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.234754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.235250 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.235251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.235424 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.235920 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.235932 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.237492 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.239333 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.239341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.239343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.239575 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.239827 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.239828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.239829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.239942 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.240286 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.240293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.240432 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.240568 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.240731 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":118,"threads_started":1}
I20260927 11:12:18.241565 8019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:12:18.249728 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.241877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34070)
0927 11:12:18.241969 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.241980 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.242048 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.242414 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.242415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.242575 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.242638 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.242640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.243522 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.243523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.244335 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.244337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.244867 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.244868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.244946 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.245317 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.245322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.246707 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.248719 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.248720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.248720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.248890 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.249106 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.249106 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.249107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.249167 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.249496 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.249497 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.249540 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.249595 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.249670 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 11:12:18.249833 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.241940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.17.95.253:33691)
0927 11:12:18.242238 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.242242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.242267 (+ 25us) server_negotiation.cc:407] Connection header received
0927 11:12:18.242290 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.242292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.242307 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.242692 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.242717 (+ 25us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.243453 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.244410 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.244782 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.244843 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.246788 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.246795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.246796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.246802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.248628 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.248975 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.248978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.248978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.249039 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.249233 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.249234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.249234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.249421 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.249484 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.249622 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.249725 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260927 11:12:18.250424 8019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:18.250746 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.250632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50412)
0927 11:12:18.250687 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.250729 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 11:12:18.250877 7969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
I20260927 11:12:18.251361 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.251237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50414)
0927 11:12:18.251300 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.251332 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 11:12:18.251564 8019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:41155: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.95.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab.
Entry for principal kudu/127.17.95.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab.
I20260927 11:12:18.258039 17791 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWCkVYM/build/release/bin/kudu
/tmp/dist-test-taskWCkVYM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:41155
--webserver_interface=127.17.95.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab
--principal=kudu/127.17.95.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.95.212:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.95.254:40307,127.17.95.253:33691,127.17.95.252:41155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 11:12:18.285709 7944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } attempt: 3
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:12:18.292757 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.286102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34086)
0927 11:12:18.286161 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.286184 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.286293 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.286463 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.286464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.286622 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.286694 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.286696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.287428 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.287430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.287954 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.287956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.288334 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.288335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.288416 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.288912 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.288916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.290246 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.291757 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.291758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.291758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.291931 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.292132 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.292133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.292133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.292199 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.292424 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.292425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.292465 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.292564 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.292673 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 11:12:18.292750 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.286183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.17.95.253:33691)
0927 11:12:18.286289 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.286292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.286302 (+ 10us) server_negotiation.cc:407] Connection header received
0927 11:12:18.286368 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.286369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.286383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.286457 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.286778 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.287334 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.288038 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.288261 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.288336 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.290330 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.290331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.290331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.290339 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.291673 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.292010 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.292012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.292012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.292064 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.292262 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.292263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.292263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.292363 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.292433 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.292587 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.292657 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 11:12:18.293378 7944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:18.293735 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.293592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50418)
0927 11:12:18.293648 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.293690 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 11:12:18.294071 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.293950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50434)
0927 11:12:18.294006 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.294041 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 11:12:18.294189 7944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:41155: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
I20260927 11:12:18.300760 8019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } attempt: 1
I20260927 11:12:18.301142 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.301014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50436)
0927 11:12:18.301066 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.301102 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 11:12:18.301513 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.301385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50444)
0927 11:12:18.301429 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.301457 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 11:12:18.301599 8019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:41155: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
I20260927 11:12:18.325351 7944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } attempt: 1
I20260927 11:12:18.325839 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.325704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50446)
0927 11:12:18.325762 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.325806 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 11:12:18.326217 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.326114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50458)
0927 11:12:18.326163 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.326194 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 11:12:18.326321 7944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:41155: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
W20260927 11:12:18.339180 8026 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 11:12:18.339377 8026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 11:12:18.339395 8026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 11:12:18.339417 8026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 11:12:18.340525 8026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 11:12:18.340569 8026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 11:12:18.340585 8026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 11:12:18.340600 8026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 11:12:18.340619 8026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 11:12:18.342175 8026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.95.212:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.95.254:40307,127.17.95.253:33691,127.17.95.252:41155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/krb5kdc/kudu_127.17.95.252.keytab
--principal=kudu/127.17.95.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.95.252:41155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.95.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 10:19:16 UTC on ccycloud.quasar-rhysdl.root.comops.site
build id 4675881
I20260927 11:12:18.342434 8026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 11:12:18.342710 8026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 11:12:18.347447 8034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:18.347456 8031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 11:12:18.347501 8032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 11:12:18.347705 8026 server_base.cc:1034] running on GCE node
I20260927 11:12:18.347954 8026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 11:12:18.348237 8026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 11:12:18.349391 8026 hybrid_clock.cc:648] HybridClock initialized: now 1790507538349376 us; error 44 us; skew 500 ppm
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 11:12:18.351676 8026 init.cc:377] Logged in from keytab as kudu/127.17.95.252@KRBTEST.COM (short username kudu)
I20260927 11:12:18.352056 8026 webserver.cc:460] Webserver started at http://127.17.95.252:41209/ using document root <none> and password file <none>
I20260927 11:12:18.352286 8026 fs_manager.cc:359] Metadata directory not provided
I20260927 11:12:18.352350 8026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 11:12:18.352511 8026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 11:12:18.353462 8026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data/instance:
uuid: "595f5ba90ab24aedb701ca02ad7d052c"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "d941784f9bf09f3b85aa019601825b5e"
server_key_iv: "08f86d0bd3bc973c3ef791d1b7033e13"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.353788 8026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal/instance:
uuid: "595f5ba90ab24aedb701ca02ad7d052c"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "d941784f9bf09f3b85aa019601825b5e"
server_key_iv: "08f86d0bd3bc973c3ef791d1b7033e13"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.355099 8026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 11:12:18.355785 8041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 11:12:18.355950 8026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 11:12:18.356041 8026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data,/tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal
uuid: "595f5ba90ab24aedb701ca02ad7d052c"
format_stamp: "Formatted at 2026-09-27 11:12:18 on dist-test-slave-6j0v"
server_key: "d941784f9bf09f3b85aa019601825b5e"
server_key_iv: "08f86d0bd3bc973c3ef791d1b7033e13"
server_key_version: "encryptionkey@0"
I20260927 11:12:18.356122 8026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 11:12:18.369745 8019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } attempt: 2
I20260927 11:12:18.370187 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.370065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50464)
0927 11:12:18.370118 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.370156 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 11:12:18.370527 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.370450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50480)
0927 11:12:18.370490 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.370511 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 11:12:18.370636 8019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.95.252:41155: Network error: Client connection negotiation failed: client connection to 127.17.95.252:41155: connect: Connection refused (error 111)
I20260927 11:12:18.373816 8026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 11:12:18.375532 8026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 11:12:18.375669 8026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 11:12:18.379307 8026 rpc_server.cc:307] RPC server started. Bound to: 127.17.95.252:41155
I20260927 11:12:18.379377 8093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.95.252:41155 every 8 connection(s)
I20260927 11:12:18.379740 8026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/data/info.pb
I20260927 11:12:18.380620 8094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 11:12:18.382035 8094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:18.382611 17791 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWCkVYM/build/release/bin/kudu as pid 8026
I20260927 11:12:18.382740 17791 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWCkVYM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790507277271108-17791-0/minicluster-data/master-2/wal/instance
I20260927 11:12:18.382942 17791 external_mini_cluster.cc:1428] Setting key f36b5265b1dab511af802bbc2ba87174
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:12:18.391784 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.382673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35566 (local address 127.17.95.254:40307)
0927 11:12:18.382714 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.382716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.382961 (+ 245us) server_negotiation.cc:407] Connection header received
0927 11:12:18.383711 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.383713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.383726 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.383772 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.384287 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.384759 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.385910 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.386104 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.386190 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.388447 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.388448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.388449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.388455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.390257 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.390669 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.390670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.390670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.390727 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.391009 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.391009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.391009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.391173 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.391233 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.391585 (+ 352us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.391670 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:12:18.391908 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.382660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.254:40307 (local address 127.0.0.1:35566)
0927 11:12:18.382841 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.382886 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.383593 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.383801 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.383806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.384048 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.384198 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.384201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.384849 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.384850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.385816 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.385819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.386219 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.386220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.386336 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.386827 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.386837 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.388370 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.390333 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.390337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.390339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.390592 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.390793 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.390794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.390795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.390936 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.391241 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.391247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.391416 (+ 169us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.391514 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.391671 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":153,"thread_start_us":94,"threads_started":1}
I20260927 11:12:18.392340 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.383420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.17.95.252:41155)
0927 11:12:18.383569 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.383574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.383609 (+ 35us) server_negotiation.cc:407] Connection header received
0927 11:12:18.383682 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.383688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.383707 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.383823 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.384502 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.385351 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.386137 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.386441 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.386535 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.389061 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.389068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.389069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.389075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.391026 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.391416 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.391419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.391419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.391473 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.391727 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.391728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.391728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.391901 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.392071 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.392151 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.392222 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260927 11:12:18.392426 8094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.254@KRBTEST.COM
I20260927 11:12:18.400875 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.392756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.253:33691 (local address 127.0.0.1:34100)
0927 11:12:18.392786 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.392797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.392836 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.393009 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.393010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.393188 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.393228 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.393230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.394128 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.394129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.394856 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.394858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.395316 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.395317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.395359 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.395739 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.395744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.397779 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.399612 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.399613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.399613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.399858 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.400136 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.400136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.400137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.400213 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.400542 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.400543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.400584 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.400648 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.400750 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260927 11:12:18.400889 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.392810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.17.95.253:33691)
0927 11:12:18.392856 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.392857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.392869 (+ 12us) server_negotiation.cc:407] Connection header received
0927 11:12:18.392894 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.392895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.392922 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.392984 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.393321 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.394046 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.394923 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.395251 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.395317 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.397867 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.397869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.397869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.397876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.399530 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.399941 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.399943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.399943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.400029 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.400315 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.400316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.400316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.400470 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.400542 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.400688 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.400769 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:12:18.401422 8094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } has no permanent_uuid. Determining permanent_uuid...
I20260927 11:12:18.401456 7944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } attempt: 2
I20260927 11:12:18.402658 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.393171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.17.95.254:40307)
0927 11:12:18.393212 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.393213 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.393222 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:12:18.393290 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.393291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.393301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.393350 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.393889 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.394559 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.395449 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.395740 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.395826 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.398644 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.398645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.398645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.398651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.401470 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.401834 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.401835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.401835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.401889 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.402116 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.402116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.402117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.402256 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.402440 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.402482 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.402550 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.252@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.254@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.95.253@KRBTEST.COM
I20260927 11:12:18.409874 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.401654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50496)
0927 11:12:18.401687 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.401698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.401736 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.401949 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.401950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.402099 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.402141 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.402143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.403175 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.403176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.403958 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.403960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.404811 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.404811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.404872 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.405286 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.405291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.406869 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.408775 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.408776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.408776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.408952 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.409150 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.409151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.409152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.409203 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.409514 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.409515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.409561 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.409654 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.409743 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 11:12:18.410313 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.401711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50496 (local address 127.17.95.252:41155)
0927 11:12:18.401842 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.401844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.401851 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:12:18.401868 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.401869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.401878 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.401932 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.402218 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.403070 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.404043 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.404710 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.404806 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.406968 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.406969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.406970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.406977 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.408678 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.409040 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.409041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.409041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.409094 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.409294 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.409295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.409295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.409451 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.409515 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.410148 (+ 633us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.410221 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260927 11:12:18.410615 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.401805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:50508)
0927 11:12:18.401962 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.401970 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.401993 (+ 23us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.402281 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.402282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.402417 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.402449 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.402451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.403347 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.403348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.404113 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.404115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.404663 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.404664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.404716 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.405107 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.405112 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.407205 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.409261 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.409262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.409262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.409495 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.409896 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.409897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.409897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.410071 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.410359 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.410360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.410395 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.410454 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.410527 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":1}
I20260927 11:12:18.410933 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.401837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50508 (local address 127.17.95.252:41155)
0927 11:12:18.402182 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.402184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.402191 (+ 7us) server_negotiation.cc:407] Connection header received
0927 11:12:18.402210 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.402211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.402221 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.402281 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.402521 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.403273 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.404191 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.404540 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.404605 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.407307 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.407308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.407309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.407315 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.409183 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.409558 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.409559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.409559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.409607 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.410166 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.410166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.410167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.410278 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.410337 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.410791 (+ 454us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.410844 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":97,"threads_started":1}
I20260927 11:12:18.412243 8094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: Bootstrap starting.
I20260927 11:12:18.412276 7944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27: Bootstrap starting.
I20260927 11:12:18.412735 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.403298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.17.95.253:33691)
0927 11:12:18.403329 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.403331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.404130 (+ 799us) server_negotiation.cc:407] Connection header received
0927 11:12:18.404194 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.404195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.404203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.404261 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.404843 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.405510 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.406162 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.406474 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.406550 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.409467 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.409468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.409469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.409476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.411543 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.411933 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.411934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.411935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.411996 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.412212 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.412213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.412213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.412343 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.412507 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.412559 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.412646 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 11:12:18.412983 8094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:18.413000 7944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:18.413297 8094 log.cc:824] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:18.413311 7944 log.cc:824] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:18.414053 8094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: No bootstrap required, opened a new log
I20260927 11:12:18.414057 17791 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 11:12:18.414204 7944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27: No bootstrap required, opened a new log
I20260927 11:12:18.415005 7918 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ad54ed5e3472409fb2ab9f99aa690c27"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35568
I20260927 11:12:18.415848 8094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.415979 8094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595f5ba90ab24aedb701ca02ad7d052c, State: Initialized, Role: FOLLOWER
I20260927 11:12:18.416064 8094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.416144 7944 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.416281 7944 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad54ed5e3472409fb2ab9f99aa690c27, State: Initialized, Role: FOLLOWER
I20260927 11:12:18.416380 8094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:18.416311 8102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.416402 7944 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.416543 8102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.416790 7944 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:18.416850 7918 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 11:12:18.416895 7918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:18.417869 7918 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.418138 7918 leader_election.cc:288] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [CANDIDATE]: Term 1 election: Requested vote from peers efb020b941a642be8f57b9b4a63e9741 (127.17.95.253:33691), 595f5ba90ab24aedb701ca02ad7d052c (127.17.95.252:41155)
I20260927 11:12:18.416771 8103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.418299 8103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.418637 8068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "595f5ba90ab24aedb701ca02ad7d052c"
I20260927 11:12:18.418730 8068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [term 0 FOLLOWER]: Advancing to term 1
W20260927 11:12:18.419361 8119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:18.419456 8119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:18.420152 8068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad54ed5e3472409fb2ab9f99aa690c27 in term 1.
I20260927 11:12:18.420372 7894 leader_election.cc:302] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595f5ba90ab24aedb701ca02ad7d052c, ad54ed5e3472409fb2ab9f99aa690c27; no voters:
I20260927 11:12:18.420485 8103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 11:12:18.420655 8103 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [term 1 LEADER]: Becoming Leader. State: Replica: ad54ed5e3472409fb2ab9f99aa690c27, State: Running, Role: LEADER
I20260927 11:12:18.420744 8103 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.421200 8128 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad54ed5e3472409fb2ab9f99aa690c27. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.421398 8128 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:18.420339 8026 master_runner.cc:186] Error getting master registration for 127.17.95.254:40307: OK, instance_id { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" instance_seqno: 1790507538059935 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 11:12:18.422848 8026 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 11:12:18.423918 8136 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 11:12:18.424094 8136 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 11:12:18.424587 7876 master_runner.cc:186] Error getting master registration for 127.17.95.253:33691: OK, instance_id { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" instance_seqno: 1790507538225950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 11:12:18.427781 7876 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 11:12:18.431694 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.419275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.17.95.254:40307)
0927 11:12:18.419322 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.419324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.419967 (+ 643us) server_negotiation.cc:407] Connection header received
0927 11:12:18.420072 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.420073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.420084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.420148 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.421790 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.422466 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.424744 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.425054 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.425121 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.428762 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.428764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.428764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.428771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.430350 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.430806 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.430807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.430808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.430867 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.431095 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.431096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.431096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.431217 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.431300 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.431520 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.431593 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 11:12:18.432231 8130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 11:12:18.432413 8130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 11:12:18.433734 8068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [term 1 FOLLOWER]: Refusing update from remote peer ad54ed5e3472409fb2ab9f99aa690c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:12:18.434027 8128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:12:18.435381 8102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad54ed5e3472409fb2ab9f99aa690c27. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.435490 8102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.438329 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.419831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.17.95.252:41155)
0927 11:12:18.419861 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.419863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.428853 (+ 8990us) server_negotiation.cc:407] Connection header received
0927 11:12:18.428973 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.428973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.428987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.429020 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.429730 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.430457 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.431452 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.431749 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.431798 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.434789 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.434791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.434791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.434811 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.436718 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.437313 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.437314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.437315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.437366 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.437639 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.437639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.437640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.437782 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.437864 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.438167 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.438245 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260927 11:12:18.440325 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.419631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.17.95.253:33691)
0927 11:12:18.419667 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.419669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.423720 (+ 4051us) server_negotiation.cc:407] Connection header received
0927 11:12:18.423824 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.423825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.423835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.423863 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.425517 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.426166 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.428198 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.428487 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.428569 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.431009 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.431010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.431011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.431029 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.433129 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.439344 (+ 6215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.439345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.439346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.439413 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.439686 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.439687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.439688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.439835 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.439912 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.440162 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.440236 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:12:18.448174 8103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.448262 8103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:18.448318 8102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.448400 8102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.448400 8138 mvcc.cc:205] Tried to move back new op lower bound from 7333918877422395392 to 7333918877372080128. Current Snapshot: MvccSnapshot[applied={T|T < 7333918877422395392}]
I20260927 11:12:18.448819 8130 catalog_manager.cc:1347] Generated new cluster ID: 9349e7c779a94de884eb9629b0a28859
I20260927 11:12:18.448884 8130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 11:12:18.461777 8019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } attempt: 3
Sep 27 11:12:18 dist-test-slave-6j0v krb5kdc[7863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790507538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.95.253@KRBTEST.COM for kudu/127.17.95.252@KRBTEST.COM
I20260927 11:12:18.470670 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.462019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.95.252:41155 (local address 127.0.0.1:57348)
0927 11:12:18.462092 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0927 11:12:18.462102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 11:12:18.462146 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 11:12:18.462490 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 11:12:18.462491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 11:12:18.462671 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 11:12:18.462716 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.462717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.464195 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.464196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.464972 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 11:12:18.464974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.465450 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.465451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 11:12:18.465502 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.465947 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 11:12:18.465952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 11:12:18.467477 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 11:12:18.469372 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.469373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.469373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.469572 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.469803 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 11:12:18.469803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 11:12:18.469804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 11:12:18.469854 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 11:12:18.470346 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 11:12:18.470347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 11:12:18.470397 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 11:12:18.470475 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 11:12:18.470569 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 11:12:18.470945 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.462294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.17.95.252:41155)
0927 11:12:18.462328 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.462330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.462336 (+ 6us) server_negotiation.cc:407] Connection header received
0927 11:12:18.462367 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.462367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.462394 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.462439 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.463390 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.464093 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.465053 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.465335 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.465387 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.467571 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 11:12:18.467572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 11:12:18.467572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 11:12:18.467579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 11:12:18.469284 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.469676 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.469677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.469677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.469740 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 11:12:18.469929 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 11:12:18.469930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 11:12:18.469930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 11:12:18.470287 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 11:12:18.470349 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.470778 (+ 429us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.470850 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:12:18.472434 8019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: Bootstrap starting.
I20260927 11:12:18.473156 8019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: Neither blocks nor log segments found. Creating new log.
I20260927 11:12:18.473460 8019 log.cc:824] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: Log is configured to *not* fsync() on all Append() calls
I20260927 11:12:18.473766 8130 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 11:12:18.474472 8130 catalog_manager.cc:1504] Loading token signing keys...
I20260927 11:12:18.477908 8019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: No bootstrap required, opened a new log
I20260927 11:12:18.478477 8130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27: Generated new TSK 0
I20260927 11:12:18.479429 8130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 11:12:18.479996 8019 raft_consensus.cc:348] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.480147 8019 raft_consensus.cc:729] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb020b941a642be8f57b9b4a63e9741, State: Initialized, Role: FOLLOWER
I20260927 11:12:18.480232 8019 consensus_queue.cc:260] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } }
I20260927 11:12:18.480527 8019 sys_catalog.cc:564] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 11:12:18.480612 7992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [term 0 FOLLOWER]: Advancing to term 1
I20260927 11:12:18.481420 7992 raft_consensus.cc:1261] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [term 1 FOLLOWER]: Refusing update from remote peer ad54ed5e3472409fb2ab9f99aa690c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 11:12:18.481710 8128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 11:12:18.482004 7993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "efb020b941a642be8f57b9b4a63e9741"
I20260927 11:12:18.482720 7993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad54ed5e3472409fb2ab9f99aa690c27 in term 1.
I20260927 11:12:18.485073 8144 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.485172 8144 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.485219 8144 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.485262 8144 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.485412 8155 catalog_manager.cc:1259] Loaded cluster ID: 9349e7c779a94de884eb9629b0a28859
I20260927 11:12:18.485458 8155 catalog_manager.cc:1552] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: loading cluster ID for follower catalog manager: success
I20260927 11:12:18.485378 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad54ed5e3472409fb2ab9f99aa690c27. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.485623 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 11:12:18.486337 8155 catalog_manager.cc:1574] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: acquiring CA information for follower catalog manager: success
I20260927 11:12:18.486631 8155 catalog_manager.cc:1602] T 00000000000000000000000000000000 P efb020b941a642be8f57b9b4a63e9741: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 11:12:18.515931 7947 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.513199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.17.95.254:40307)
0927 11:12:18.513244 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.513246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.515293 (+ 2047us) server_negotiation.cc:407] Connection header received
0927 11:12:18.515399 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.515400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.515412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.515443 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.515863 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55804: BlockingRecv error: recv got EOF from 127.0.0.1:55804 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 11:12:18.516682 8023 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.513536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.17.95.253:33691)
0927 11:12:18.513573 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.513575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.516129 (+ 2554us) server_negotiation.cc:407] Connection header received
0927 11:12:18.516223 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.516225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.516235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.516263 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.516612 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44928: BlockingRecv error: recv got EOF from 127.0.0.1:44928 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 11:12:18.517403 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.513806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.17.95.252:41155)
0927 11:12:18.513845 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.513846 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.516901 (+ 3055us) server_negotiation.cc:407] Connection header received
0927 11:12:18.517018 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.517019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.517030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.517059 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.517343 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57364: BlockingRecv error: recv got EOF from 127.0.0.1:57364 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 11:12:18.526808 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.523334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.17.95.253:33691)
0927 11:12:18.523368 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.523370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.523551 (+ 181us) server_negotiation.cc:407] Connection header received
0927 11:12:18.523662 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.523663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.523702 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.523744 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.523938 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.524723 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.525845 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.526168 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.526211 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.526427 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.526551 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.526598 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.526721 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.526738 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 11:12:18.529834 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.529123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.17.95.252:41155)
0927 11:12:18.529159 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.529160 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.529341 (+ 181us) server_negotiation.cc:407] Connection header received
0927 11:12:18.529438 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.529439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.529455 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.529483 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.529787 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57378: BlockingRecv error: recv got EOF from 127.0.0.1:57378 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:18.535157 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.532004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.17.95.253:33691)
0927 11:12:18.532036 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.532038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.532260 (+ 222us) server_negotiation.cc:407] Connection header received
0927 11:12:18.532347 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.532348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.532356 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.532382 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.532531 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.533254 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.534329 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.534618 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.534665 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.534847 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.534878 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.534904 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.535002 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.535017 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 11:12:18.538007 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.537297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.17.95.252:41155)
0927 11:12:18.537331 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.537333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.537526 (+ 193us) server_negotiation.cc:407] Connection header received
0927 11:12:18.537616 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.537617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.537627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.537668 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.537959 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57394: BlockingRecv error: recv got EOF from 127.0.0.1:57394 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:18.544229 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.541041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.17.95.253:33691)
0927 11:12:18.541074 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.541075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.541292 (+ 217us) server_negotiation.cc:407] Connection header received
0927 11:12:18.541384 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.541385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.541393 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.541419 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.541595 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.542328 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.543483 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.543785 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.543821 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.544000 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.544029 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.544054 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.544151 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.544167 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 11:12:18.547001 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.546306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.17.95.252:41155)
0927 11:12:18.546339 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.546341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.546523 (+ 182us) server_negotiation.cc:407] Connection header received
0927 11:12:18.546611 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.546612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.546621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.546662 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.546954 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57398: BlockingRecv error: recv got EOF from 127.0.0.1:57398 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 11:12:18.555168 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.552123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.17.95.253:33691)
0927 11:12:18.552158 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.552159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.552380 (+ 221us) server_negotiation.cc:407] Connection header received
0927 11:12:18.552468 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.552469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.552477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.552504 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.552668 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.553368 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.554427 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.554714 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.554750 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.554923 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.554952 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.554976 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.555092 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.555108 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 11:12:18.557967 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.557245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.17.95.252:41155)
0927 11:12:18.557289 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.557291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.557485 (+ 194us) server_negotiation.cc:407] Connection header received
0927 11:12:18.557573 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.557574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.557583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.557610 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.557918 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57414: BlockingRecv error: recv got EOF from 127.0.0.1:57414 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:12:18.570531 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.567243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.17.95.253:33691)
0927 11:12:18.567283 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.567285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.567562 (+ 277us) server_negotiation.cc:407] Connection header received
0927 11:12:18.567668 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.567669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.567679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.567707 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.567884 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.568630 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.569758 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.570038 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.570075 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.570259 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.570292 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.570319 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.570419 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.570439 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 11:12:18.573460 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.572721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.17.95.252:41155)
0927 11:12:18.572758 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.572760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.572952 (+ 192us) server_negotiation.cc:407] Connection header received
0927 11:12:18.573045 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.573046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.573057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.573084 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.573407 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57430: BlockingRecv error: recv got EOF from 127.0.0.1:57430 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 11:12:18.594044 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.590671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44968 (local address 127.17.95.253:33691)
0927 11:12:18.590713 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.590715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.590984 (+ 269us) server_negotiation.cc:407] Connection header received
0927 11:12:18.591127 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.591128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.591138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.591170 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.591353 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.592091 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.593202 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.593483 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.593527 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.593748 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.593782 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.593811 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.593955 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.593972 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260927 11:12:18.597395 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.596604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.17.95.252:41155)
0927 11:12:18.596654 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.596656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.596862 (+ 206us) server_negotiation.cc:407] Connection header received
0927 11:12:18.596957 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.596958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.596969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.596998 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.597342 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57440: BlockingRecv error: recv got EOF from 127.0.0.1:57440 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 11:12:18.634184 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.630661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.17.95.253:33691)
0927 11:12:18.630707 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.630709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.631015 (+ 306us) server_negotiation.cc:407] Connection header received
0927 11:12:18.631155 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.631156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.631168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.631198 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.631384 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.632120 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.633312 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.633632 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.633736 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.633917 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.633954 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.634004 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.634090 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.634111 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 11:12:18.636920 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.636069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.17.95.252:41155)
0927 11:12:18.636129 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.636131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.636354 (+ 223us) server_negotiation.cc:407] Connection header received
0927 11:12:18.636396 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.636397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.636411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.636480 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.636863 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57456: BlockingRecv error: recv got EOF from 127.0.0.1:57456 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 11:12:18.704075 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.701945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.17.95.253:33691)
0927 11:12:18.702002 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.702004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.702111 (+ 107us) server_negotiation.cc:407] Connection header received
0927 11:12:18.702192 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.702192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.702202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.702242 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.702345 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.702839 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.703567 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.703749 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.703798 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.703906 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.703930 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.703962 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.704007 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.704019 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260927 11:12:18.706382 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.705697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.17.95.252:41155)
0927 11:12:18.705781 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.705783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.705814 (+ 31us) server_negotiation.cc:407] Connection header received
0927 11:12:18.705920 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.705921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.705932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.705999 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.706318 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57472: BlockingRecv error: recv got EOF from 127.0.0.1:57472 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 11:12:18.837770 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:18.835509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.17.95.253:33691)
0927 11:12:18.835572 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.835574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.835702 (+ 128us) server_negotiation.cc:407] Connection header received
0927 11:12:18.835735 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.835736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.835746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.835811 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:18.835930 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.836431 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.837190 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:18.837368 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:18.837429 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:18.837555 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:18.837582 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:18.837622 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:18.837691 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 11:12:18.837707 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260927 11:12:18.840065 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:18.839516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57476 (local address 127.17.95.252:41155)
0927 11:12:18.839579 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:18.839581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:18.839640 (+ 59us) server_negotiation.cc:407] Connection header received
0927 11:12:18.839674 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:18.839675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:18.839685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:18.839734 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:18.839988 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57476: BlockingRecv error: recv got EOF from 127.0.0.1:57476 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:12:18.980662 8128 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad54ed5e3472409fb2ab9f99aa690c27" member_type: VOTER last_known_addr { host: "127.17.95.254" port: 40307 } } peers { permanent_uuid: "efb020b941a642be8f57b9b4a63e9741" member_type: VOTER last_known_addr { host: "127.17.95.253" port: 33691 } } peers { permanent_uuid: "595f5ba90ab24aedb701ca02ad7d052c" member_type: VOTER last_known_addr { host: "127.17.95.252" port: 41155 } } }
I20260927 11:12:18.980813 8128 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad54ed5e3472409fb2ab9f99aa690c27 [sys.catalog]: This master's current role is: LEADER
I20260927 11:12:19.099637 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:19.097218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45008 (local address 127.17.95.253:33691)
0927 11:12:19.097268 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.097270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.097381 (+ 111us) server_negotiation.cc:407] Connection header received
0927 11:12:19.097424 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.097425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.097437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.097514 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:19.097682 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.098170 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.098907 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.099114 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.099185 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:19.099350 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:19.099382 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:19.099421 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:19.099499 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 11:12:19.099516 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260927 11:12:19.101908 8099 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 11:12:19.101372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.17.95.252:41155)
0927 11:12:19.101446 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.101448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.101457 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:12:19.101483 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.101484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.101495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.101538 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 11:12:19.101854 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57490: BlockingRecv error: recv got EOF from 127.0.0.1:57490 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 11:12:19.419924 8119 catalog_manager.cc:1259] Loaded cluster ID: 9349e7c779a94de884eb9629b0a28859
I20260927 11:12:19.419994 8119 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: loading cluster ID for follower catalog manager: success
I20260927 11:12:19.420828 8119 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: acquiring CA information for follower catalog manager: success
I20260927 11:12:19.421092 8119 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 595f5ba90ab24aedb701ca02ad7d052c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 11:12:19.617556 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:19.614943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.17.95.253:33691)
0927 11:12:19.615062 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.615064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.615111 (+ 47us) server_negotiation.cc:407] Connection header received
0927 11:12:19.615189 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.615189 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.615200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.615254 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:19.615434 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.615901 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.616827 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.617003 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.617097 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:19.617304 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:19.617343 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:19.617381 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:19.617435 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 11:12:19.617452 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260927 11:12:19.621320 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:19.619176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.17.95.252:41155)
0927 11:12:19.619265 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.619267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.619275 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:12:19.619331 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.619331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.619384 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.619435 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:19.619511 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.619938 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.620535 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.620725 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.620776 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:19.620959 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:19.621071 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:19.621132 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:19.621174 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 11:12:19.621201 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":49,"threads_started":1}
I20260927 11:12:19.625677 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:19.623193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.17.95.252:41155)
0927 11:12:19.623235 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.623237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.623327 (+ 90us) server_negotiation.cc:407] Connection header received
0927 11:12:19.623386 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.623387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.623397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.623468 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:19.623632 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.624312 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.625026 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.625248 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.625302 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:19.625458 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:19.625484 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:19.625528 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:19.625612 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 11:12:19.625632 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 11:12:19.626884 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:19.623095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.17.95.253:33691)
0927 11:12:19.623144 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.623145 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.623154 (+ 9us) server_negotiation.cc:407] Connection header received
0927 11:12:19.623186 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.623187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.623197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.623252 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:19.623456 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.624193 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.626202 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.626369 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.626413 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:19.626596 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:19.626637 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:19.626711 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:19.626774 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 11:12:19.626795 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 11:12:19.626978 8293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 11:12:19.622960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.17.95.254:40307)
0927 11:12:19.623266 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 11:12:19.623269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 11:12:19.623277 (+ 8us) server_negotiation.cc:407] Connection header received
0927 11:12:19.623332 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 11:12:19.623333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 11:12:19.623384 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 11:12:19.623520 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 11:12:19.624427 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.625190 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.626201 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 11:12:19.626389 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 11:12:19.626503 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 11:12:19.626679 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 11:12:19.626758 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 11:12:19.626829 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 11:12:19.626880 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 11:12:19.626900 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":55,"threads_started":1}
I20260927 11:12:19.633558 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7876
I20260927 11:12:19.638692 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 7951
I20260927 11:12:19.643011 17791 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWCkVYM/build/release/bin/kudu with pid 8026
2026-09-27T11:12:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2821 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5573 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (262379 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST